You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "haohuaijin (via GitHub)" <gi...@apache.org> on 2026/07/27 02:58:10 UTC

[PR] fix(arrow-buffer): preserve bits outside the requested range in in-place bitwise ops [arrow-rs]

haohuaijin opened a new pull request, #10444:
URL: https://github.com/apache/arrow-rs/pull/10444

   # Which issue does this PR close?
   
   No dedicated issue — this was found while implementing #10425, whose optimization is the first caller that depends on the behaviour fixed here. Happy to file one if you'd prefer to track it separately.
   
   # Rationale for this change
   
   `apply_bitwise_binary_op` and `apply_bitwise_unary_op` are built around the invariant that only the bits in `offset_in_bits..offset_in_bits + len_in_bits` are modified. That intent is stated in the doc comments of the helpers implementing it — `align_to_byte` ("while preserving all other bits in the byte"), `set_remainder_bits` ("leaving all other bits in the affected bytes unchanged") and `handle_mutable_buffer_remainder_unary` ("ensuring that bits outside the operation range are preserved") — but two paths did not honour it.
   
   **1. `align_to_byte` ignored `len_in_bits`.** It wrote every bit from `bit_offset` up to the byte boundary. When the requested range both started and ended inside the same non-byte-aligned byte, the trailing bits of that byte were overwritten with `op` applied to padding:
   
   ```rust
   let mut left = vec![0b11111111u8, 0b11111111u8];
   let right = vec![0b00000000u8, 0b00000000u8];
   // AND a single bit at offset 1
   apply_bitwise_binary_op(&mut left, 1, &right, 0, 1, |a, b| a & b);
   // expected [0b11111101, 0b11111111], got [0b00000001, 0b11111111]
   ```
   
   **2. `set_remainder_bits` dropped the boundary byte's out-of-range bits.** It read that byte into the *low* bits of a `u64` and masked it with `!((1 << remainder_len) - 1)`. Whenever the remainder spanned more than one byte the mask selected nothing, so those bits were zeroed rather than preserved. This affected any call whose length left a remainder of 9..63 bits that is not a multiple of 8.
   
   ## No existing caller was affected
   
   I checked all four in-tree callers, which is why this went unnoticed:
   
   | Caller | Why it is immune |
   | --- | --- |
   | `BooleanBufferBuilder::append_packed_range` | Copies with `\|_a, b\| b` into capacity just zeroed by `advance()`; both bugs force out-of-range bits to `0`, which is what they already were |
   | `BooleanBuffer::bitwise_bin_op_assign` | Runs only on a uniquely owned buffer (`into_mutable()`) and re-wraps the result with the original offset/length, so out-of-range bits are unobservable |
   | `BooleanArray::try_bitwise_unary_in_place` | Same uniquely-owned pattern |
   | `BooleanArray::try_bitwise_bin_op_in_place` | Same uniquely-owned pattern |
   
   # What changes are included in this PR?
   
   - `align_to_byte` takes `len_in_bits` and masks its write to `bit_offset..bit_offset + min(8 - bit_offset, len_in_bits)`.
   - `set_remainder_bits` shifts the boundary byte to the position it actually occupies within the word before masking.
   - The guarantee is now stated on both public functions, since callers that read the surrounding bits back rely on it.
   
   # Are these changes tested?
   
   Yes. The root cause of the gap was that the two shared test helpers only asserted the bits *inside* the operated range, so out-of-range corruption was invisible to every test using them. Both helpers now also assert that every bit outside the range is byte-for-byte identical before and after, which retroactively covers all of their existing call sites — that alone surfaced bug 2 in 11 pre-existing tests.
   
   Added on top of that:
   
   - `test_ops_ending_inside_the_first_partial_byte` — sweeps every `offset`/`len` pair contained in one partial byte, for AND/OR/XOR and NOT, including differing left/right offsets.
   - `test_and_within_first_partial_byte_preserves_trailing_bits` and `test_not_within_first_partial_byte_preserves_trailing_bits` — minimal regression tests for the two reproducers.
   
   `cargo test -p arrow-buffer` passes (342 unit + 54 doc tests), as do `arrow-array`, `arrow` and `arrow-select`, which exercise the fixed helpers through the callers above. Clippy and fmt are clean.
   
   # Are there any user-facing changes?
   
   A behaviour change in two public functions, in the direction of the documented intent: bits outside the requested range are no longer clobbered. No API signature changes and no breaking changes — as shown above, no existing caller could observe the old behaviour. The preservation guarantee is now explicit in the rustdoc for both functions.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(arrow-buffer): preserve bits outside the requested range in in-place bitwise ops [arrow-rs]

Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey commented on PR #10444:
URL: https://github.com/apache/arrow-rs/pull/10444#issuecomment-5117771170

   run benchmark builder
   env:
     BENCH_FILTER: bench_bool


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(arrow-buffer): preserve bits outside the requested range in in-place bitwise ops [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb merged PR #10444:
URL: https://github.com/apache/arrow-rs/pull/10444


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(arrow-buffer): preserve bits outside the requested range in in-place bitwise ops [arrow-rs]

Posted by "haohuaijin (via GitHub)" <gi...@apache.org>.
haohuaijin commented on PR #10444:
URL: https://github.com/apache/arrow-rs/pull/10444#issuecomment-5173547873

   Thanks @Jefffrey @alamb 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(arrow-buffer): preserve bits outside the requested range in in-place bitwise ops [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #10444:
URL: https://github.com/apache/arrow-rs/pull/10444#issuecomment-5117801679

   🤖 Arrow criterion benchmark running (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/10444#issuecomment-5117771170)
   **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c5117771170-1287-t5q5v 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux`
   <details><summary>CPU Details (lscpu)</summary>
   
   ```
   Architecture:                            aarch64
   CPU op-mode(s):                          64-bit
   Byte Order:                              Little Endian
   CPU(s):                                  16
   On-line CPU(s) list:                     0-15
   Vendor ID:                               ARM
   Model name:                              Neoverse-V2
   Model:                                   1
   Thread(s) per core:                      1
   Core(s) per cluster:                     16
   Socket(s):                               -
   Cluster(s):                              1
   Stepping:                                r0p1
   BogoMIPS:                                2000.00
   Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
   L1d cache:                               1 MiB (16 instances)
   L1i cache:                               1 MiB (16 instances)
   L2 cache:                                32 MiB (16 instances)
   L3 cache:                                80 MiB (1 instance)
   NUMA node(s):                            1
   NUMA node0 CPU(s):                       0-15
   Vulnerability Gather data sampling:      Not affected
   Vulnerability Indirect target selection: Not affected
   Vulnerability Itlb multihit:             Not affected
   Vulnerability L1tf:                      Not affected
   Vulnerability Mds:                       Not affected
   Vulnerability Meltdown:                  Not affected
   Vulnerability Mmio stale data:           Not affected
   Vulnerability Reg file data sampling:    Not affected
   Vulnerability Retbleed:                  Not affected
   Vulnerability Spec rstack overflow:      Not affected
   Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
   Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
   Vulnerability Spectre v2:                Mitigation; CSV2, BHB
   Vulnerability Srbds:                     Not affected
   Vulnerability Tsa:                       Not affected
   Vulnerability Tsx async abort:           Not affected
   Vulnerability Vmscape:                   Not affected
   ```
   
   </details>
   
   Comparing fix-inplace-bitwise-out-of-range-bits (3f4c0c2f90be72f65bcefe178841c2b23e9e3114) to 005a2fd (merge-base) [diff](https://github.com/apache/arrow-rs/compare/005a2fd43f1b243ed43f4cfa356ebe8f0757939b..3f4c0c2f90be72f65bcefe178841c2b23e9e3114)
   BENCH_NAME=builder
   BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental,object_store --bench builder
   BENCH_FILTER=bench_bool
   Results will be posted here when complete
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(arrow-buffer): preserve bits outside the requested range in in-place bitwise ops [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #10444:
URL: https://github.com/apache/arrow-rs/pull/10444#issuecomment-5117851223

   🤖 Arrow criterion benchmark completed (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/10444#issuecomment-5117808630)
   
   **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
   
   <details><summary>CPU Details (lscpu)</summary>
   
   ```
   Architecture:                            aarch64
   CPU op-mode(s):                          64-bit
   Byte Order:                              Little Endian
   CPU(s):                                  16
   On-line CPU(s) list:                     0-15
   Vendor ID:                               ARM
   Model name:                              Neoverse-V2
   Model:                                   1
   Thread(s) per core:                      1
   Core(s) per cluster:                     16
   Socket(s):                               -
   Cluster(s):                              1
   Stepping:                                r0p1
   BogoMIPS:                                2000.00
   Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
   L1d cache:                               1 MiB (16 instances)
   L1i cache:                               1 MiB (16 instances)
   L2 cache:                                32 MiB (16 instances)
   L3 cache:                                80 MiB (1 instance)
   NUMA node(s):                            1
   NUMA node0 CPU(s):                       0-15
   Vulnerability Gather data sampling:      Not affected
   Vulnerability Indirect target selection: Not affected
   Vulnerability Itlb multihit:             Not affected
   Vulnerability L1tf:                      Not affected
   Vulnerability Mds:                       Not affected
   Vulnerability Meltdown:                  Not affected
   Vulnerability Mmio stale data:           Not affected
   Vulnerability Reg file data sampling:    Not affected
   Vulnerability Retbleed:                  Not affected
   Vulnerability Spec rstack overflow:      Not affected
   Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
   Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
   Vulnerability Spectre v2:                Mitigation; CSV2, BHB
   Vulnerability Srbds:                     Not affected
   Vulnerability Tsa:                       Not affected
   Vulnerability Tsx async abort:           Not affected
   Vulnerability Vmscape:                   Not affected
   ```
   
   </details>
   
   <details><summary>Details</summary>
   <p>
   
   ```
   group                    fix-inplace-bitwise-out-of-range-bits    main
   -----                    -------------------------------------    ----
   bench_bool/bench_bool    1.00    361.3±0.18µs  1383.8 MB/sec      1.00    361.4±0.14µs  1383.5 MB/sec
   ```
   
   </p>
   </details>
   
   <details><summary>Resource Usage</summary>
   
   **base (merge-base)**
   | Metric | Value |
   |--------|-------|
   | Wall time | 15.0s |
   | Peak memory | 4.3 MiB |
   | Avg memory | 3.1 MiB |
   | CPU user | 10.8s |
   | CPU sys | 0.0s |
   | Peak spill | 0 B |
   
   **branch**
   | Metric | Value |
   |--------|-------|
   | Wall time | 15.0s |
   | Peak memory | 4.2 MiB |
   | Avg memory | 2.9 MiB |
   | CPU user | 10.0s |
   | CPU sys | 0.0s |
   | Peak spill | 0 B |
   </details>
   
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(arrow-buffer): preserve bits outside the requested range in in-place bitwise ops [arrow-rs]

Posted by "haohuaijin (via GitHub)" <gi...@apache.org>.
haohuaijin commented on code in PR #10444:
URL: https://github.com/apache/arrow-rs/pull/10444#discussion_r3704947650


##########
arrow-buffer/src/util/bit_util.rs:
##########
@@ -468,12 +487,16 @@ where
     // 4. Shift back the result to the original position
     let result_first_byte = result_first_byte << bit_offset;
 
-    // 5. Mask the bits that are outside the relevant bits in the byte
-    //    so the bits until bit_offset are 1 and the rest are 0
-    let mask_for_first_bit_offset = (1 << bit_offset) - 1;
+    // 5. Mask in only the bits the caller asked to process, i.e. the bits in
+    //    `bit_offset..bit_offset + bits_in_this_byte`. The request may end before the
+    //    byte boundary, in which case the trailing bits must be preserved as well.
+    //
+    //    `bits_in_this_byte + bit_offset <= 8`, so the mask always fits in a `u8`. The
+    //    shift is done in `u16` only so the expression stays correct for a full byte.

Review Comment:
   Good catch, updated in [f86947c](https://github.com/apache/arrow-rs/pull/10444/commits/f86947cc8f6fbde7dec43b1089a13e918aabdf0d), use `u8` instead of `u16`, becuase the bit_offset only can in the range 1-7.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(arrow-buffer): preserve bits outside the requested range in in-place bitwise ops [arrow-rs]

Posted by "haohuaijin (via GitHub)" <gi...@apache.org>.
haohuaijin commented on PR #10444:
URL: https://github.com/apache/arrow-rs/pull/10444#issuecomment-5087672144

   While implementing #10425 I hit corrupted mask tails, and traced it to two bugs in the in-place bitwise helpers — both write outside `offset..offset+len`, which they're documented not to do:
   
   - `align_to_byte` ignored `len_in_bits`, so when the range started and ended inside one non-byte-aligned byte, it overwrote the rest of that byte.
   - `set_remainder_bits` read the boundary byte into the low bits of a `u64` before masking, so once the remainder spanned more than one byte its out-of-range bits were zeroed instead of preserved.
   
   cc @alamb 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(arrow-buffer): preserve bits outside the requested range in in-place bitwise ops [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #10444:
URL: https://github.com/apache/arrow-rs/pull/10444#issuecomment-5117842630

   🤖 Arrow criterion benchmark running (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/10444#issuecomment-5117808630)
   **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB) | `Linux bench-c5117808630-1288-rv58z 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux`
   <details><summary>CPU Details (lscpu)</summary>
   
   ```
   Architecture:                            aarch64
   CPU op-mode(s):                          64-bit
   Byte Order:                              Little Endian
   CPU(s):                                  16
   On-line CPU(s) list:                     0-15
   Vendor ID:                               ARM
   Model name:                              Neoverse-V2
   Model:                                   1
   Thread(s) per core:                      1
   Core(s) per cluster:                     16
   Socket(s):                               -
   Cluster(s):                              1
   Stepping:                                r0p1
   BogoMIPS:                                2000.00
   Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
   L1d cache:                               1 MiB (16 instances)
   L1i cache:                               1 MiB (16 instances)
   L2 cache:                                32 MiB (16 instances)
   L3 cache:                                80 MiB (1 instance)
   NUMA node(s):                            1
   NUMA node0 CPU(s):                       0-15
   Vulnerability Gather data sampling:      Not affected
   Vulnerability Indirect target selection: Not affected
   Vulnerability Itlb multihit:             Not affected
   Vulnerability L1tf:                      Not affected
   Vulnerability Mds:                       Not affected
   Vulnerability Meltdown:                  Not affected
   Vulnerability Mmio stale data:           Not affected
   Vulnerability Reg file data sampling:    Not affected
   Vulnerability Retbleed:                  Not affected
   Vulnerability Spec rstack overflow:      Not affected
   Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
   Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
   Vulnerability Spectre v2:                Mitigation; CSV2, BHB
   Vulnerability Srbds:                     Not affected
   Vulnerability Tsa:                       Not affected
   Vulnerability Tsx async abort:           Not affected
   Vulnerability Vmscape:                   Not affected
   ```
   
   </details>
   
   Comparing fix-inplace-bitwise-out-of-range-bits (3f4c0c2f90be72f65bcefe178841c2b23e9e3114) to 005a2fd (merge-base) [diff](https://github.com/apache/arrow-rs/compare/005a2fd43f1b243ed43f4cfa356ebe8f0757939b..3f4c0c2f90be72f65bcefe178841c2b23e9e3114)
   BENCH_NAME=builder
   BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental,object_store --bench builder
   BENCH_FILTER=bench_bool
   Results will be posted here when complete
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(arrow-buffer): preserve bits outside the requested range in in-place bitwise ops [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #10444:
URL: https://github.com/apache/arrow-rs/pull/10444#issuecomment-5117818955

   🤖 Arrow criterion benchmark completed (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/10444#issuecomment-5117771170)
   
   **Instance:** `c4a-highmem-16` (12 vCPU / 65 GiB)
   
   <details><summary>CPU Details (lscpu)</summary>
   
   ```
   Architecture:                            aarch64
   CPU op-mode(s):                          64-bit
   Byte Order:                              Little Endian
   CPU(s):                                  16
   On-line CPU(s) list:                     0-15
   Vendor ID:                               ARM
   Model name:                              Neoverse-V2
   Model:                                   1
   Thread(s) per core:                      1
   Core(s) per cluster:                     16
   Socket(s):                               -
   Cluster(s):                              1
   Stepping:                                r0p1
   BogoMIPS:                                2000.00
   Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
   L1d cache:                               1 MiB (16 instances)
   L1i cache:                               1 MiB (16 instances)
   L2 cache:                                32 MiB (16 instances)
   L3 cache:                                80 MiB (1 instance)
   NUMA node(s):                            1
   NUMA node0 CPU(s):                       0-15
   Vulnerability Gather data sampling:      Not affected
   Vulnerability Indirect target selection: Not affected
   Vulnerability Itlb multihit:             Not affected
   Vulnerability L1tf:                      Not affected
   Vulnerability Mds:                       Not affected
   Vulnerability Meltdown:                  Not affected
   Vulnerability Mmio stale data:           Not affected
   Vulnerability Reg file data sampling:    Not affected
   Vulnerability Retbleed:                  Not affected
   Vulnerability Spec rstack overflow:      Not affected
   Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
   Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
   Vulnerability Spectre v2:                Mitigation; CSV2, BHB
   Vulnerability Srbds:                     Not affected
   Vulnerability Tsa:                       Not affected
   Vulnerability Tsx async abort:           Not affected
   Vulnerability Vmscape:                   Not affected
   ```
   
   </details>
   
   <details><summary>Details</summary>
   <p>
   
   ```
   group                    fix-inplace-bitwise-out-of-range-bits    main
   -----                    -------------------------------------    ----
   bench_bool/bench_bool    1.00    361.3±0.35µs  1384.0 MB/sec      1.00    361.4±0.18µs  1383.5 MB/sec
   ```
   
   </p>
   </details>
   
   <details><summary>Resource Usage</summary>
   
   **base (merge-base)**
   | Metric | Value |
   |--------|-------|
   | Wall time | 15.0s |
   | Peak memory | 4.3 MiB |
   | Avg memory | 3.1 MiB |
   | CPU user | 10.7s |
   | CPU sys | 0.0s |
   | Peak spill | 0 B |
   
   **branch**
   | Metric | Value |
   |--------|-------|
   | Wall time | 15.0s |
   | Peak memory | 4.3 MiB |
   | Avg memory | 2.9 MiB |
   | CPU user | 10.0s |
   | CPU sys | 0.0s |
   | Peak spill | 0 B |
   </details>
   
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(arrow-buffer): preserve bits outside the requested range in in-place bitwise ops [arrow-rs]

Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey commented on code in PR #10444:
URL: https://github.com/apache/arrow-rs/pull/10444#discussion_r3702556932


##########
arrow-buffer/src/util/bit_util.rs:
##########
@@ -468,12 +487,16 @@ where
     // 4. Shift back the result to the original position
     let result_first_byte = result_first_byte << bit_offset;
 
-    // 5. Mask the bits that are outside the relevant bits in the byte
-    //    so the bits until bit_offset are 1 and the rest are 0
-    let mask_for_first_bit_offset = (1 << bit_offset) - 1;
+    // 5. Mask in only the bits the caller asked to process, i.e. the bits in
+    //    `bit_offset..bit_offset + bits_in_this_byte`. The request may end before the
+    //    byte boundary, in which case the trailing bits must be preserved as well.
+    //
+    //    `bits_in_this_byte + bit_offset <= 8`, so the mask always fits in a `u8`. The
+    //    shift is done in `u16` only so the expression stays correct for a full byte.

Review Comment:
   would we ever mask to a full byte? i believe the assert above would protect against that? or is this just for safety



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(arrow-buffer): preserve bits outside the requested range in in-place bitwise ops [arrow-rs]

Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey commented on PR #10444:
URL: https://github.com/apache/arrow-rs/pull/10444#issuecomment-5117808630

   run benchmark builder
   env:
     BENCH_FILTER: bench_bool


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org