You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "alamb (via GitHub)" <gi...@apache.org> on 2025/11/21 17:52:30 UTC

[PR] WIP: Boolean mask backed RowSelection [arrow-rs]

alamb opened a new pull request, #8902:
URL: https://github.com/apache/arrow-rs/pull/8902

   # Which issue does this PR close?
   
   We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax.
   
   - Closes https://github.com/apache/arrow-rs/issues/8844
   
   # Rationale for this change
   
   Make the parquet predicate evaluation faster by not converting back/forth between BooleanArray and RowSelection as much
   
   # What changes are included in this PR?
   1. Change RowSelection to have two possible backings
   2. Add a BooleanArray backed implementation, based on @XiangpengHao 's code from https://github.com/apache/arrow-rs/pull/6624
   
   # Are these changes tested?
   TBD
   
   
   # Are there any user-facing changes?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] WIP: Boolean mask backed RowSelection [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #8902:
URL: https://github.com/apache/arrow-rs/pull/8902#discussion_r2560363367


##########
parquet/src/arrow/arrow_reader/selection/mask.rs:
##########
@@ -0,0 +1,566 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use crate::arrow::arrow_reader::{RowSelection, RowSelector};
+use arrow_array::{Array, BooleanArray};
+use arrow_buffer::bit_iterator::BitIndexIterator;
+use arrow_buffer::{BooleanBuffer, BooleanBufferBuilder, MutableBuffer};
+use std::ops::Range;
+
+/// Selection based on a [`BooleanArray`]
+///
+/// This represents the result of a filter evaluation as a bitmap. It is
+/// more efficient for sparse filter results with many small skips or selections.
+///
+/// It is similar to the "Bitmap Index" described in
+/// [Predicate Caching: Query-Driven Secondary Indexing for Cloud Data Warehouses]
+///
+/// [Predicate Caching: Query-Driven Secondary Indexing for Cloud Data Warehouses]: https://dl.acm.org/doi/10.1145/3626246.3653395
+///
+/// Originally from Xiangpeng Hao in <https://github.com/apache/arrow-rs/pull/6624>
+#[derive(Debug, Clone, Eq, PartialEq)]
+pub(crate) struct BooleanRowSelection {
+    selector: BooleanBuffer,
+}
+
+impl BooleanRowSelection {
+    /// Create a new [`BooleanRowSelection] from a list of [`BooleanArray`].
+    pub fn from_filters(filters: &[BooleanArray]) -> Self {
+        let arrays: Vec<&dyn Array> = filters.iter().map(|x| x as &dyn Array).collect();
+        let result = arrow_select::concat::concat(&arrays).unwrap().into_data();
+        let (boolean_array, _null) = BooleanArray::from(result).into_parts();
+        BooleanRowSelection {
+            selector: boolean_array,
+        }
+    }
+
+    /// Create a new [`BooleanRowSelection`] with all rows unselected
+    pub fn new_unselected(row_count: usize) -> Self {
+        let buffer = BooleanBuffer::new_unset(row_count);
+
+        BooleanRowSelection { selector: buffer }
+    }
+
+    /// Create a new [`BooleanRowSelection`] with all rows selected
+    pub fn new_selected(row_count: usize) -> Self {
+        let buffer = BooleanBuffer::new_set(row_count);
+
+        BooleanRowSelection { selector: buffer }
+    }
+
+    /// Returns a new [`BooleanRowSelection`] that selects the inverse of this [`BooleanRowSelection`].
+    pub fn as_inverted(&self) -> Self {
+        let buffer = !&self.selector;
+        BooleanRowSelection { selector: buffer }
+    }
+
+    /// Returns the number of rows selected by this [`BooleanRowSelection`].
+    pub fn row_count(&self) -> usize {
+        self.selector.count_set_bits()
+    }
+
+    /// Create a new [`BooleanRowSelection`] from a list of consecutive ranges.
+    pub fn from_consecutive_ranges(
+        ranges: impl Iterator<Item = Range<usize>>,
+        total_rows: usize,
+    ) -> Self {
+        let mut buffer = BooleanBufferBuilder::new(total_rows);
+        let mut last_end = 0;
+
+        for range in ranges {
+            let len = range.end - range.start;
+            if len == 0 {
+                continue;
+            }
+
+            if range.start > last_end {
+                buffer.append_n(range.start - last_end, false);
+            }
+            buffer.append_n(len, true);
+            last_end = range.end;
+        }
+
+        if last_end != total_rows {
+            buffer.append_n(total_rows - last_end, false);
+        }
+
+        BooleanRowSelection {
+            selector: buffer.finish(),
+        }
+    }
+
+    /// Compute the union of two [`BooleanRowSelection`]
+    /// For example:
+    /// self:      NNYYYYNNYYNYN
+    /// other:     NYNNNNNNN
+    ///
+    /// returned:  NYYYYYNNYYNYN
+    #[must_use]
+    pub fn union(&self, other: &Self) -> Self {
+        // use arrow::compute::kernels::boolean::or;
+
+        let union_selectors = &self.selector | &other.selector;
+
+        BooleanRowSelection {
+            selector: union_selectors,
+        }
+    }
+
+    /// Compute the intersection of two [`BooleanRowSelection`]
+    /// For example:
+    /// self:      NNYYYYNNYYNYN
+    /// other:     NYNNNNNNY
+    ///
+    /// returned:  NNNNNNNNYYNYN
+    #[must_use]
+    pub fn intersection(&self, other: &Self) -> Self {
+        let intersection_selectors = &self.selector & &other.selector;
+
+        BooleanRowSelection {
+            selector: intersection_selectors,
+        }
+    }
+
+    /// Combines this [`BooleanRowSelection`] with another using logical AND on the selected bits.
+    ///
+    /// Unlike intersection, the `other` [`BooleanRowSelection`] must have exactly as many set bits as `self`.
+    /// This method will keep only the bits in `self` that are also set in `other`
+    /// at the positions corresponding to `self`'s set bits.
+    pub fn and_then(&self, other: &Self) -> Self {
+        // Ensure that 'other' has exactly as many set bits as 'self'
+        debug_assert_eq!(
+            self.row_count(),
+            other.selector.len(),
+            "The 'other' selection must have exactly as many set bits as 'self'."
+        );
+
+        if self.selector.len() == other.selector.len() {
+            // fast path if the two selections are the same length
+            // common if this is the first predicate
+            debug_assert_eq!(self.row_count(), self.selector.len());
+            return self.intersection(other);
+        }
+
+        let mut buffer = MutableBuffer::from_len_zeroed(self.selector.inner().len());
+        buffer.copy_from_slice(self.selector.inner().as_slice());
+        let mut builder = BooleanBufferBuilder::new_from_buffer(buffer, self.selector.len());
+
+        // Create iterators for 'self' and 'other' bits
+        let mut other_bits = other.selector.iter();
+
+        for bit_idx in self.true_iter() {
+            let predicate = other_bits
+                .next()
+                .expect("Mismatch in set bits between self and other");
+            if !predicate {
+                builder.set_bit(bit_idx, false);
+            }
+        }
+
+        BooleanRowSelection {
+            selector: builder.finish(),
+        }
+    }
+
+    /// Returns an iterator over the indices of the set bits in this [`BooleanRowSelection`]
+    pub fn true_iter(&self) -> BitIndexIterator<'_> {
+        self.selector.set_indices()
+    }
+
+    /// Returns `true` if this [`BooleanRowSelection`] selects any rows
+    pub fn selects_any(&self) -> bool {
+        self.true_iter().next().is_some()
+    }
+
+    /// Returns a new [`BooleanRowSelection`] that selects the rows in this [`BooleanRowSelection`] from `offset` to `offset + len`
+    pub fn slice(&self, offset: usize, len: usize) -> BooleanArray {
+        BooleanArray::new(self.selector.slice(offset, len), None)
+    }
+}
+
+impl From<Vec<RowSelector>> for BooleanRowSelection {
+    fn from(selection: Vec<RowSelector>) -> Self {
+        let selection = RowSelection::from(selection);
+        RowSelection::into(selection)
+    }
+}
+
+impl From<RowSelection> for BooleanRowSelection {
+    fn from(selection: RowSelection) -> Self {
+        let total_rows = selection.row_count();
+        let mut builder = BooleanBufferBuilder::new(total_rows);
+
+        for selector in selection.iter() {
+            if selector.skip {
+                builder.append_n(selector.row_count, false);
+            } else {
+                builder.append_n(selector.row_count, true);
+            }
+        }
+
+        BooleanRowSelection {
+            selector: builder.finish(),
+        }
+    }
+}
+
+impl From<&BooleanRowSelection> for RowSelection {
+    fn from(selection: &BooleanRowSelection) -> Self {
+        let array = BooleanArray::new(selection.selector.clone(), None);
+        RowSelection::from_filters(&[array])
+    }
+}
+
+/// Combines this [`BooleanBuffer`] with another using logical AND on the selected bits.
+///
+/// Unlike intersection, the `other` [`BooleanBuffer`] must have exactly as many **set bits** as `self`,
+/// i.e., self.count_set_bits() == other.len().
+///
+/// This method will keep only the bits in `self` that are also set in `other`
+/// at the positions corresponding to `self`'s set bits.
+/// For example:
+/// left:   NNYYYNNYYNYN
+/// right:    YNY  NY N
+/// result: NNYNYNNNYNNN
+///
+/// Optimized version of `boolean_buffer_and_then` using BMI2 PDEP instructions.
+/// This function performs the same operation but uses bit manipulation instructions
+/// for better performance on supported x86_64 CPUs.
+pub fn boolean_buffer_and_then(left: &BooleanBuffer, right: &BooleanBuffer) -> BooleanBuffer {

Review Comment:
   @XiangpengHao did this it in liquid cache:
   - https://github.com/XiangpengHao/liquid-cache/blob/60323cdb5f17f88af633ad9acbc7d74f684f9912/src/parquet/src/utils.rs#L17
   
   
   I will copy that new version over here in this PR
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] WIP: Boolean mask backed RowSelection [arrow-rs]

Posted by "rluvaton (via GitHub)" <gi...@apache.org>.
rluvaton commented on code in PR #8902:
URL: https://github.com/apache/arrow-rs/pull/8902#discussion_r2578619571


##########
parquet/src/arrow/arrow_reader/selection/and_then.rs:
##########
@@ -0,0 +1,347 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Optimized [`boolean_buffer_and_then`] kernel, based on code by
+//! Xiangpeng Hao in <https://github.com/apache/arrow-rs/pull/6624> and
+//! [LiquidCache's implementation](https://github.com/XiangpengHao/liquid-cache/blob/60323cdb5f17f88af633ad9acbc7d74f684f9912/src/parquet/src/utils.rs#L17)
+
+use crate::arrow::arrow_reader::RowSelector;
+use arrow_buffer::{BooleanBuffer, BooleanBufferBuilder, MutableBuffer};
+
+fn boolean_buffer_and_then_fallback(left: &BooleanBuffer, right: &BooleanBuffer) -> BooleanBuffer {
+    debug_assert_eq!(
+        left.count_set_bits(),
+        right.len(),
+        "the right selection must have the same number of set bits as the left selection"

Review Comment:
   you check left set bits, not right 
   ```suggestion
           "the left selection must have the same number of set bits as the right selection"
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] WIP: Boolean mask backed RowSelection [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #8902:
URL: https://github.com/apache/arrow-rs/pull/8902#issuecomment-3649240457

   run benchmark arrow_reader


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] WIP: Boolean mask backed RowSelection [arrow-rs]

Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #8902:
URL: https://github.com/apache/arrow-rs/pull/8902#issuecomment-3647146400

   🤖 `./gh_compare_arrow.sh` [gh_compare_arrow.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/gh_compare_arrow.sh) Running
   Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
   Comparing alamb/sketch_boolean_selection (b77a7eb290fb4b7d5225c53eae8f63dc65d80b6a) to 08dcc0b8c552e81d612b53bc0ab2a0007bfe20a1 [diff](https://github.com/apache/arrow-rs/compare/08dcc0b8c552e81d612b53bc0ab2a0007bfe20a1..b77a7eb290fb4b7d5225c53eae8f63dc65d80b6a)
   BENCH_NAME=arrow_reader_clickbench
   BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental --bench arrow_reader_clickbench 
   BENCH_FILTER=
   BENCH_BRANCH_NAME=alamb_sketch_boolean_selection
   Results will be posted here when complete
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] WIP: Boolean mask backed RowSelection [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed pull request #8902: WIP: Boolean mask backed RowSelection
URL: https://github.com/apache/arrow-rs/pull/8902


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] WIP: Boolean mask backed RowSelection [arrow-rs]

Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #8902:
URL: https://github.com/apache/arrow-rs/pull/8902#issuecomment-3647367180

   🤖 `./gh_compare_arrow.sh` [gh_compare_arrow.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/gh_compare_arrow.sh) Running
   Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
   Comparing alamb/sketch_boolean_selection (b2686039b76eeb09fbf814d76e7ef53d7a8bf7f8) to 08dcc0b8c552e81d612b53bc0ab2a0007bfe20a1 [diff](https://github.com/apache/arrow-rs/compare/08dcc0b8c552e81d612b53bc0ab2a0007bfe20a1..b2686039b76eeb09fbf814d76e7ef53d7a8bf7f8)
   BENCH_NAME=arrow_reader_clickbench
   BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental --bench arrow_reader_clickbench 
   BENCH_FILTER=
   BENCH_BRANCH_NAME=alamb_sketch_boolean_selection
   Results will be posted here when complete
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] WIP: Boolean mask backed RowSelection [arrow-rs]

Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #8902:
URL: https://github.com/apache/arrow-rs/pull/8902#issuecomment-3647455202

   🤖: Benchmark completed
   
   <details><summary>Details</summary>
   <p>
   
   
   ```
   group                                alamb_sketch_boolean_selection         main
   -----                                ------------------------------         ----
   arrow_reader_clickbench/async/Q1     1.01      2.4±0.03ms        ? ?/sec    1.00      2.3±0.04ms        ? ?/sec
   arrow_reader_clickbench/async/Q10    1.05     14.3±0.35ms        ? ?/sec    1.00     13.7±0.31ms        ? ?/sec
   arrow_reader_clickbench/async/Q11    1.06     16.2±0.33ms        ? ?/sec    1.00     15.3±0.31ms        ? ?/sec
   arrow_reader_clickbench/async/Q12    1.09     29.3±0.71ms        ? ?/sec    1.00     26.9±0.47ms        ? ?/sec
   arrow_reader_clickbench/async/Q13    1.09     34.9±0.56ms        ? ?/sec    1.00     32.1±0.33ms        ? ?/sec
   arrow_reader_clickbench/async/Q14    1.08     32.0±0.63ms        ? ?/sec    1.00     29.5±0.38ms        ? ?/sec
   arrow_reader_clickbench/async/Q19    1.01      5.6±0.12ms        ? ?/sec    1.00      5.5±0.09ms        ? ?/sec
   arrow_reader_clickbench/async/Q20    1.25    151.2±2.69ms        ? ?/sec    1.00    121.3±0.58ms        ? ?/sec
   arrow_reader_clickbench/async/Q21    1.10    172.0±1.89ms        ? ?/sec    1.00    155.9±2.43ms        ? ?/sec
   arrow_reader_clickbench/async/Q22    1.07   321.2±29.69ms        ? ?/sec    1.00   299.0±20.59ms        ? ?/sec
   arrow_reader_clickbench/async/Q23    1.02    416.1±5.02ms        ? ?/sec    1.00    406.9±6.95ms        ? ?/sec
   arrow_reader_clickbench/async/Q24    1.13     39.6±1.04ms        ? ?/sec    1.00     35.2±0.52ms        ? ?/sec
   arrow_reader_clickbench/async/Q27    1.04    103.0±0.79ms        ? ?/sec    1.00     98.9±0.76ms        ? ?/sec
   arrow_reader_clickbench/async/Q28    1.06    103.6±1.74ms        ? ?/sec    1.00     97.8±0.74ms        ? ?/sec
   arrow_reader_clickbench/async/Q30    1.14     36.5±0.41ms        ? ?/sec    1.00     32.0±0.39ms        ? ?/sec
   arrow_reader_clickbench/async/Q36    1.06    114.5±1.28ms        ? ?/sec    1.00    107.8±0.86ms        ? ?/sec
   arrow_reader_clickbench/async/Q37    1.07     90.4±1.23ms        ? ?/sec    1.00     84.7±1.17ms        ? ?/sec
   arrow_reader_clickbench/async/Q38    1.07     34.1±0.53ms        ? ?/sec    1.00     31.8±0.47ms        ? ?/sec
   arrow_reader_clickbench/async/Q39    1.06     47.6±1.18ms        ? ?/sec    1.00     45.0±0.52ms        ? ?/sec
   arrow_reader_clickbench/async/Q40    1.15     30.4±0.64ms        ? ?/sec    1.00     26.6±0.68ms        ? ?/sec
   arrow_reader_clickbench/async/Q41    1.13     24.2±1.19ms        ? ?/sec    1.00     21.4±0.29ms        ? ?/sec
   arrow_reader_clickbench/async/Q42    1.10     11.2±0.35ms        ? ?/sec    1.00     10.2±0.12ms        ? ?/sec
   arrow_reader_clickbench/sync/Q1      1.21      2.6±0.05ms        ? ?/sec    1.00      2.2±0.03ms        ? ?/sec
   arrow_reader_clickbench/sync/Q10     1.07     10.7±0.30ms        ? ?/sec    1.00     10.0±0.14ms        ? ?/sec
   arrow_reader_clickbench/sync/Q11     1.04     12.3±0.24ms        ? ?/sec    1.00     11.9±0.18ms        ? ?/sec
   arrow_reader_clickbench/sync/Q12     1.16     41.6±0.97ms        ? ?/sec    1.00     35.8±0.79ms        ? ?/sec
   arrow_reader_clickbench/sync/Q13     1.02     51.6±0.86ms        ? ?/sec    1.00     50.8±1.67ms        ? ?/sec
   arrow_reader_clickbench/sync/Q14     1.02     46.7±0.84ms        ? ?/sec    1.00     45.9±0.40ms        ? ?/sec
   arrow_reader_clickbench/sync/Q19     1.24      5.7±0.07ms        ? ?/sec    1.00      4.6±0.08ms        ? ?/sec
   arrow_reader_clickbench/sync/Q20     1.01    179.8±1.25ms        ? ?/sec    1.00    178.1±2.09ms        ? ?/sec
   arrow_reader_clickbench/sync/Q21     1.00    237.8±2.82ms        ? ?/sec    1.01    241.1±4.91ms        ? ?/sec
   arrow_reader_clickbench/sync/Q22     1.01    486.2±4.03ms        ? ?/sec    1.00    481.2±8.11ms        ? ?/sec
   arrow_reader_clickbench/sync/Q23     1.01   446.1±15.24ms        ? ?/sec    1.00   443.6±13.63ms        ? ?/sec
   arrow_reader_clickbench/sync/Q24     1.05     47.8±0.52ms        ? ?/sec    1.00     45.8±0.73ms        ? ?/sec
   arrow_reader_clickbench/sync/Q27     1.02    156.8±1.45ms        ? ?/sec    1.00    153.4±1.28ms        ? ?/sec
   arrow_reader_clickbench/sync/Q28     1.02    152.3±1.33ms        ? ?/sec    1.00    149.3±2.33ms        ? ?/sec
   arrow_reader_clickbench/sync/Q30     1.03     33.9±0.39ms        ? ?/sec    1.00     32.8±0.44ms        ? ?/sec
   arrow_reader_clickbench/sync/Q36     1.07    150.6±1.38ms        ? ?/sec    1.00    141.2±1.07ms        ? ?/sec
   arrow_reader_clickbench/sync/Q37     1.10     85.6±1.10ms        ? ?/sec    1.00     77.5±0.64ms        ? ?/sec
   arrow_reader_clickbench/sync/Q38     1.19     29.7±0.22ms        ? ?/sec    1.00     24.9±0.24ms        ? ?/sec
   arrow_reader_clickbench/sync/Q39     1.11     35.4±0.39ms        ? ?/sec    1.00     31.9±0.35ms        ? ?/sec
   arrow_reader_clickbench/sync/Q40     1.18     28.8±0.56ms        ? ?/sec    1.00     24.4±0.31ms        ? ?/sec
   arrow_reader_clickbench/sync/Q41     1.25     24.6±0.55ms        ? ?/sec    1.00     19.7±0.49ms        ? ?/sec
   arrow_reader_clickbench/sync/Q42     1.36     12.8±0.11ms        ? ?/sec    1.00      9.4±0.12ms        ? ?/sec
   ```
   
   
   </p>
   </details>
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] WIP: Boolean mask backed RowSelection [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #8902:
URL: https://github.com/apache/arrow-rs/pull/8902#issuecomment-3649240400

   run benchmark arrow_reader_clickbench


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] WIP: Boolean mask backed RowSelection [arrow-rs]

Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #8902:
URL: https://github.com/apache/arrow-rs/pull/8902#issuecomment-3647142364

   🤖 `./gh_compare_arrow.sh` [gh_compare_arrow.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/gh_compare_arrow.sh) Running
   Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
   Comparing alamb/sketch_boolean_selection (b77a7eb290fb4b7d5225c53eae8f63dc65d80b6a) to 08dcc0b8c552e81d612b53bc0ab2a0007bfe20a1 [diff](https://github.com/apache/arrow-rs/compare/08dcc0b8c552e81d612b53bc0ab2a0007bfe20a1..b77a7eb290fb4b7d5225c53eae8f63dc65d80b6a)
   BENCH_NAME=arrow_reader
   BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental --bench arrow_reader 
   BENCH_FILTER=
   BENCH_BRANCH_NAME=alamb_sketch_boolean_selection
   Results will be posted here when complete
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] WIP: Boolean mask backed RowSelection [arrow-rs]

Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #8902:
URL: https://github.com/apache/arrow-rs/pull/8902#issuecomment-3647147149

   🤖 `./gh_compare_arrow.sh` [gh_compare_arrow.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/gh_compare_arrow.sh) Running
   Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
   Comparing alamb/sketch_boolean_selection (b77a7eb290fb4b7d5225c53eae8f63dc65d80b6a) to 08dcc0b8c552e81d612b53bc0ab2a0007bfe20a1 [diff](https://github.com/apache/arrow-rs/compare/08dcc0b8c552e81d612b53bc0ab2a0007bfe20a1..b77a7eb290fb4b7d5225c53eae8f63dc65d80b6a)
   BENCH_NAME=arrow_reader_row_filter
   BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental --bench arrow_reader_row_filter 
   BENCH_FILTER=
   BENCH_BRANCH_NAME=alamb_sketch_boolean_selection
   Results will be posted here when complete
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] WIP: Boolean mask backed RowSelection [arrow-rs]

Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #8902:
URL: https://github.com/apache/arrow-rs/pull/8902#issuecomment-3647203426

   🤖 `./gh_compare_arrow.sh` [gh_compare_arrow.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/gh_compare_arrow.sh) Running
   Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
   Comparing alamb/sketch_boolean_selection (bb5661f959247abfb44f785277f2b22ebfd6d14a) to 08dcc0b8c552e81d612b53bc0ab2a0007bfe20a1 [diff](https://github.com/apache/arrow-rs/compare/08dcc0b8c552e81d612b53bc0ab2a0007bfe20a1..bb5661f959247abfb44f785277f2b22ebfd6d14a)
   BENCH_NAME=arrow_reader
   BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental --bench arrow_reader 
   BENCH_FILTER=
   BENCH_BRANCH_NAME=alamb_sketch_boolean_selection
   Results will be posted here when complete
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] WIP: Boolean mask backed RowSelection [arrow-rs]

Posted by "rluvaton (via GitHub)" <gi...@apache.org>.
rluvaton commented on code in PR #8902:
URL: https://github.com/apache/arrow-rs/pull/8902#discussion_r2578695552


##########
parquet/src/arrow/arrow_reader/selection/and_then.rs:
##########
@@ -0,0 +1,347 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Optimized [`boolean_buffer_and_then`] kernel, based on code by
+//! Xiangpeng Hao in <https://github.com/apache/arrow-rs/pull/6624> and
+//! [LiquidCache's implementation](https://github.com/XiangpengHao/liquid-cache/blob/60323cdb5f17f88af633ad9acbc7d74f684f9912/src/parquet/src/utils.rs#L17)
+
+use crate::arrow::arrow_reader::RowSelector;
+use arrow_buffer::{BooleanBuffer, BooleanBufferBuilder, MutableBuffer};
+
+fn boolean_buffer_and_then_fallback(left: &BooleanBuffer, right: &BooleanBuffer) -> BooleanBuffer {
+    debug_assert_eq!(
+        left.count_set_bits(),
+        right.len(),
+        "the right selection must have the same number of set bits as the left selection"
+    );
+
+    if left.len() == right.len() {
+        debug_assert_eq!(left.count_set_bits(), left.len());
+        return right.clone();
+    }
+
+    let mut buffer = MutableBuffer::from_len_zeroed(left.values().len());
+    buffer.copy_from_slice(left.values());
+    let mut builder = BooleanBufferBuilder::new_from_buffer(buffer, left.len());

Review Comment:
   Note that you have a bug here and also copy more data than needed.
   
   `BooleanBuffer` have an offset to start from the `values` and here you:
   1. add more data than needed by creating a buffer with everything
   2. the left set_indices here need to be offseted by the `left.offset()` in order to set the correct bit (you should fix to not copy unneeded values and then you can keep this) 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] WIP: Boolean mask backed RowSelection [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #8902:
URL: https://github.com/apache/arrow-rs/pull/8902#issuecomment-3647202154

   run benchmark arrow_reader_clickbench


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] WIP: Boolean mask backed RowSelection [arrow-rs]

Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #8902:
URL: https://github.com/apache/arrow-rs/pull/8902#issuecomment-3649270938

   🤖: Benchmark completed
   
   <details><summary>Details</summary>
   <p>
   
   
   ```
   group                                alamb_sketch_boolean_selection         main
   -----                                ------------------------------         ----
   arrow_reader_clickbench/async/Q1     1.01      2.4±0.03ms        ? ?/sec    1.00      2.3±0.04ms        ? ?/sec
   arrow_reader_clickbench/async/Q10    1.03     13.9±0.28ms        ? ?/sec    1.00     13.5±0.34ms        ? ?/sec
   arrow_reader_clickbench/async/Q11    1.02     15.6±0.37ms        ? ?/sec    1.00     15.4±0.58ms        ? ?/sec
   arrow_reader_clickbench/async/Q12    1.08     28.8±0.46ms        ? ?/sec    1.00     26.8±0.52ms        ? ?/sec
   arrow_reader_clickbench/async/Q13    1.06     33.9±0.70ms        ? ?/sec    1.00     32.0±0.39ms        ? ?/sec
   arrow_reader_clickbench/async/Q14    1.04     31.4±0.28ms        ? ?/sec    1.00     30.1±0.39ms        ? ?/sec
   arrow_reader_clickbench/async/Q19    1.00      5.5±0.17ms        ? ?/sec    1.06      5.8±0.16ms        ? ?/sec
   arrow_reader_clickbench/async/Q20    1.01    151.2±1.69ms        ? ?/sec    1.00    149.6±1.30ms        ? ?/sec
   arrow_reader_clickbench/async/Q21    1.02    171.5±1.05ms        ? ?/sec    1.00    168.3±1.63ms        ? ?/sec
   arrow_reader_clickbench/async/Q22    1.03    330.1±9.44ms        ? ?/sec    1.00   320.4±22.03ms        ? ?/sec
   arrow_reader_clickbench/async/Q23    1.02    407.9±2.63ms        ? ?/sec    1.00    401.2±3.07ms        ? ?/sec
   arrow_reader_clickbench/async/Q24    1.05     36.9±0.43ms        ? ?/sec    1.00     35.0±0.36ms        ? ?/sec
   arrow_reader_clickbench/async/Q27    1.03    101.3±0.95ms        ? ?/sec    1.00     98.5±0.71ms        ? ?/sec
   arrow_reader_clickbench/async/Q28    1.04    100.7±0.82ms        ? ?/sec    1.00     97.1±0.89ms        ? ?/sec
   arrow_reader_clickbench/async/Q30    1.08     34.5±0.36ms        ? ?/sec    1.00     31.9±0.31ms        ? ?/sec
   arrow_reader_clickbench/async/Q36    1.03    111.0±0.73ms        ? ?/sec    1.00    107.5±0.75ms        ? ?/sec
   arrow_reader_clickbench/async/Q37    1.04     87.4±1.34ms        ? ?/sec    1.00     84.1±1.24ms        ? ?/sec
   arrow_reader_clickbench/async/Q38    1.03     32.9±0.75ms        ? ?/sec    1.00     31.8±0.32ms        ? ?/sec
   arrow_reader_clickbench/async/Q39    1.03     45.9±0.85ms        ? ?/sec    1.00     44.4±0.47ms        ? ?/sec
   arrow_reader_clickbench/async/Q40    1.06     27.6±0.32ms        ? ?/sec    1.00     26.1±0.35ms        ? ?/sec
   arrow_reader_clickbench/async/Q41    1.07     22.4±0.54ms        ? ?/sec    1.00     20.9±0.61ms        ? ?/sec
   arrow_reader_clickbench/async/Q42    1.08     10.9±0.23ms        ? ?/sec    1.00     10.0±0.19ms        ? ?/sec
   arrow_reader_clickbench/sync/Q1      1.22      2.6±0.04ms        ? ?/sec    1.00      2.1±0.01ms        ? ?/sec
   arrow_reader_clickbench/sync/Q10     1.05     10.5±0.11ms        ? ?/sec    1.00     10.0±0.43ms        ? ?/sec
   arrow_reader_clickbench/sync/Q11     1.07     12.2±0.20ms        ? ?/sec    1.00     11.4±0.13ms        ? ?/sec
   arrow_reader_clickbench/sync/Q12     1.21     41.0±0.31ms        ? ?/sec    1.00     33.8±0.28ms        ? ?/sec
   arrow_reader_clickbench/sync/Q13     1.09     51.2±0.87ms        ? ?/sec    1.00     46.7±1.37ms        ? ?/sec
   arrow_reader_clickbench/sync/Q14     1.06     46.4±0.50ms        ? ?/sec    1.00     43.6±0.47ms        ? ?/sec
   arrow_reader_clickbench/sync/Q19     1.30      5.6±0.03ms        ? ?/sec    1.00      4.3±0.07ms        ? ?/sec
   arrow_reader_clickbench/sync/Q20     1.03    178.4±1.36ms        ? ?/sec    1.00    173.6±1.86ms        ? ?/sec
   arrow_reader_clickbench/sync/Q21     1.02    235.9±2.68ms        ? ?/sec    1.00    230.2±2.61ms        ? ?/sec
   arrow_reader_clickbench/sync/Q22     1.02    481.4±2.99ms        ? ?/sec    1.00    469.9±3.41ms        ? ?/sec
   arrow_reader_clickbench/sync/Q23     1.03   440.0±15.00ms        ? ?/sec    1.00   429.0±14.53ms        ? ?/sec
   arrow_reader_clickbench/sync/Q24     1.02     47.3±0.62ms        ? ?/sec    1.00     46.3±0.49ms        ? ?/sec
   arrow_reader_clickbench/sync/Q27     1.03    155.5±0.82ms        ? ?/sec    1.00    150.7±1.61ms        ? ?/sec
   arrow_reader_clickbench/sync/Q28     1.03    150.1±1.13ms        ? ?/sec    1.00    145.2±1.55ms        ? ?/sec
   arrow_reader_clickbench/sync/Q30     1.04     33.2±0.43ms        ? ?/sec    1.00     31.9±0.65ms        ? ?/sec
   arrow_reader_clickbench/sync/Q36     1.06    148.6±0.92ms        ? ?/sec    1.00    139.9±1.10ms        ? ?/sec
   arrow_reader_clickbench/sync/Q37     1.09     84.2±0.64ms        ? ?/sec    1.00     77.2±1.15ms        ? ?/sec
   arrow_reader_clickbench/sync/Q38     1.18     29.3±0.35ms        ? ?/sec    1.00     24.8±0.34ms        ? ?/sec
   arrow_reader_clickbench/sync/Q39     1.09     34.7±0.30ms        ? ?/sec    1.00     31.8±0.45ms        ? ?/sec
   arrow_reader_clickbench/sync/Q40     1.16     28.1±0.60ms        ? ?/sec    1.00     24.3±0.65ms        ? ?/sec
   arrow_reader_clickbench/sync/Q41     1.21     23.6±0.29ms        ? ?/sec    1.00     19.5±0.30ms        ? ?/sec
   arrow_reader_clickbench/sync/Q42     1.35     12.6±0.30ms        ? ?/sec    1.00      9.4±0.17ms        ? ?/sec
   ```
   
   
   </p>
   </details>
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] WIP: Boolean mask backed RowSelection [arrow-rs]

Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #8902:
URL: https://github.com/apache/arrow-rs/pull/8902#issuecomment-3647365984

   🤖 Hi @alamb, you asked to view the benchmark queue (https://github.com/apache/arrow-rs/pull/8902#issuecomment-3647365791).
   
   | Job | User | Benchmarks | Comment |
   | --- | --- | --- | --- |
   | `arrow-8902-3647141874.sh` | alamb | arrow_reader arrow_reader_clickbench arrow_reader_row_filter | `https://github.com/apache/arrow-rs/pull/8902#issuecomment-3647141874` |
   | `arrow-8902-3647202154.sh` | alamb | arrow_reader_clickbench | `https://github.com/apache/arrow-rs/pull/8902#issuecomment-3647202154` |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] WIP: Boolean mask backed RowSelection [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #8902:
URL: https://github.com/apache/arrow-rs/pull/8902#issuecomment-3647365791

   show benchmark queue


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] WIP: Boolean mask backed RowSelection [arrow-rs]

Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #8902:
URL: https://github.com/apache/arrow-rs/pull/8902#issuecomment-3649240476

   🤖 `./gh_compare_arrow.sh` [gh_compare_arrow.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/gh_compare_arrow.sh) Running
   Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
   Comparing alamb/sketch_boolean_selection (b2686039b76eeb09fbf814d76e7ef53d7a8bf7f8) to 08dcc0b8c552e81d612b53bc0ab2a0007bfe20a1 [diff](https://github.com/apache/arrow-rs/compare/08dcc0b8c552e81d612b53bc0ab2a0007bfe20a1..b2686039b76eeb09fbf814d76e7ef53d7a8bf7f8)
   BENCH_NAME=arrow_reader_clickbench
   BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental --bench arrow_reader_clickbench 
   BENCH_FILTER=
   BENCH_BRANCH_NAME=alamb_sketch_boolean_selection
   Results will be posted here when complete
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] WIP: Boolean mask backed RowSelection [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #8902:
URL: https://github.com/apache/arrow-rs/pull/8902#issuecomment-3649467900

   I have been profiling this morning, and basically I have concluded I need to do two things: https://github.com/apache/arrow-rs/issues/8844#issuecomment-3649465923
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] WIP: Boolean mask backed RowSelection [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #8902:
URL: https://github.com/apache/arrow-rs/pull/8902#discussion_r2594006396


##########
parquet/src/arrow/arrow_reader/selection/and_then.rs:
##########
@@ -0,0 +1,347 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Optimized [`boolean_buffer_and_then`] kernel, based on code by
+//! Xiangpeng Hao in <https://github.com/apache/arrow-rs/pull/6624> and
+//! [LiquidCache's implementation](https://github.com/XiangpengHao/liquid-cache/blob/60323cdb5f17f88af633ad9acbc7d74f684f9912/src/parquet/src/utils.rs#L17)
+
+use crate::arrow::arrow_reader::RowSelector;
+use arrow_buffer::{BooleanBuffer, BooleanBufferBuilder, MutableBuffer};
+
+fn boolean_buffer_and_then_fallback(left: &BooleanBuffer, right: &BooleanBuffer) -> BooleanBuffer {
+    debug_assert_eq!(
+        left.count_set_bits(),
+        right.len(),
+        "the right selection must have the same number of set bits as the left selection"
+    );
+
+    if left.len() == right.len() {
+        debug_assert_eq!(left.count_set_bits(), left.len());
+        return right.clone();

Review Comment:
   I think in this case `left` is always (and asserted) to be all `1`s so no AND is necessary



##########
parquet/src/arrow/arrow_reader/selection/and_then.rs:
##########
@@ -0,0 +1,347 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Optimized [`boolean_buffer_and_then`] kernel, based on code by
+//! Xiangpeng Hao in <https://github.com/apache/arrow-rs/pull/6624> and
+//! [LiquidCache's implementation](https://github.com/XiangpengHao/liquid-cache/blob/60323cdb5f17f88af633ad9acbc7d74f684f9912/src/parquet/src/utils.rs#L17)
+
+use crate::arrow::arrow_reader::RowSelector;
+use arrow_buffer::{BooleanBuffer, BooleanBufferBuilder, MutableBuffer};
+
+fn boolean_buffer_and_then_fallback(left: &BooleanBuffer, right: &BooleanBuffer) -> BooleanBuffer {
+    debug_assert_eq!(
+        left.count_set_bits(),
+        right.len(),
+        "the right selection must have the same number of set bits as the left selection"
+    );
+
+    if left.len() == right.len() {
+        debug_assert_eq!(left.count_set_bits(), left.len());
+        return right.clone();
+    }
+
+    let mut buffer = MutableBuffer::from_len_zeroed(left.values().len());
+    buffer.copy_from_slice(left.values());
+    let mut builder = BooleanBufferBuilder::new_from_buffer(buffer, left.len());

Review Comment:
   This was a really good call. When I was porting over the tests they caught exactly this problem
   
   I actually found we could do the same thing, but probably faster via
   ```
       let mut builder = BooleanBufferBuilder::new(left.len());
       builder.append_buffer(&left);
   ```
   
   Which then calls `BooleanBufferBuilder::append_packed_range`
   
   (conveniently I know someone who just optimized  that function 😆 )



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] WIP: Boolean mask backed RowSelection [arrow-rs]

Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #8902:
URL: https://github.com/apache/arrow-rs/pull/8902#issuecomment-3649271095

   🤖 `./gh_compare_arrow.sh` [gh_compare_arrow.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/gh_compare_arrow.sh) Running
   Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
   Comparing alamb/sketch_boolean_selection (b2686039b76eeb09fbf814d76e7ef53d7a8bf7f8) to 08dcc0b8c552e81d612b53bc0ab2a0007bfe20a1 [diff](https://github.com/apache/arrow-rs/compare/08dcc0b8c552e81d612b53bc0ab2a0007bfe20a1..b2686039b76eeb09fbf814d76e7ef53d7a8bf7f8)
   BENCH_NAME=arrow_reader
   BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental --bench arrow_reader 
   BENCH_FILTER=
   BENCH_BRANCH_NAME=alamb_sketch_boolean_selection
   Results will be posted here when complete
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] WIP: Boolean mask backed RowSelection [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #8902:
URL: https://github.com/apache/arrow-rs/pull/8902#issuecomment-3647531438

   🤔  going the wrong direction


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] WIP: Boolean mask backed RowSelection [arrow-rs]

Posted by "rluvaton (via GitHub)" <gi...@apache.org>.
rluvaton commented on code in PR #8902:
URL: https://github.com/apache/arrow-rs/pull/8902#discussion_r2578699382


##########
parquet/src/arrow/arrow_reader/selection/and_then.rs:
##########
@@ -0,0 +1,347 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Optimized [`boolean_buffer_and_then`] kernel, based on code by
+//! Xiangpeng Hao in <https://github.com/apache/arrow-rs/pull/6624> and
+//! [LiquidCache's implementation](https://github.com/XiangpengHao/liquid-cache/blob/60323cdb5f17f88af633ad9acbc7d74f684f9912/src/parquet/src/utils.rs#L17)
+
+use crate::arrow::arrow_reader::RowSelector;
+use arrow_buffer::{BooleanBuffer, BooleanBufferBuilder, MutableBuffer};
+
+fn boolean_buffer_and_then_fallback(left: &BooleanBuffer, right: &BooleanBuffer) -> BooleanBuffer {
+    debug_assert_eq!(
+        left.count_set_bits(),
+        right.len(),
+        "the right selection must have the same number of set bits as the left selection"
+    );
+
+    if left.len() == right.len() {
+        debug_assert_eq!(left.count_set_bits(), left.len());
+        return right.clone();

Review Comment:
   don't you need to do `&` here?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] WIP: Boolean mask backed RowSelection [arrow-rs]

Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #8902:
URL: https://github.com/apache/arrow-rs/pull/8902#issuecomment-3649406409

   🤖: Benchmark completed
   
   <details><summary>Details</summary>
   <p>
   
   
   ```
   group                                                                                                      alamb_sketch_boolean_selection         main
   -----                                                                                                      ------------------------------         ----
   arrow_array_reader/BYTE_ARRAY/Decimal128Array/plain encoded, mandatory, no NULLs                           1.05  1317.3±16.11µs        ? ?/sec    1.00   1252.1±5.29µs        ? ?/sec
   arrow_array_reader/BYTE_ARRAY/Decimal128Array/plain encoded, optional, half NULLs                          1.00   1312.8±4.51µs        ? ?/sec    1.00  1310.8±17.33µs        ? ?/sec
   arrow_array_reader/BYTE_ARRAY/Decimal128Array/plain encoded, optional, no NULLs                            1.05   1321.6±4.86µs        ? ?/sec    1.00   1259.2±6.58µs        ? ?/sec
   arrow_array_reader/BinaryArray/dictionary encoded, mandatory, no NULLs                                     1.00    489.0±4.22µs        ? ?/sec    1.05   514.9±12.11µs        ? ?/sec
   arrow_array_reader/BinaryArray/dictionary encoded, optional, half NULLs                                    1.00   652.9±18.72µs        ? ?/sec    1.06    693.6±4.51µs        ? ?/sec
   arrow_array_reader/BinaryArray/dictionary encoded, optional, no NULLs                                      1.00    479.6±4.51µs        ? ?/sec    1.08    520.1±9.23µs        ? ?/sec
   arrow_array_reader/BinaryArray/plain encoded, mandatory, no NULLs                                          1.06    585.7±3.74µs        ? ?/sec    1.00    554.7±6.52µs        ? ?/sec
   arrow_array_reader/BinaryArray/plain encoded, optional, half NULLs                                         1.00    745.3±5.21µs        ? ?/sec    1.01   755.2±10.26µs        ? ?/sec
   arrow_array_reader/BinaryArray/plain encoded, optional, no NULLs                                           1.05    597.5±4.83µs        ? ?/sec    1.00    568.1±4.99µs        ? ?/sec
   arrow_array_reader/BinaryViewArray/dictionary encoded, mandatory, no NULLs                                 1.00    251.6±3.48µs        ? ?/sec    1.04    261.7±5.80µs        ? ?/sec
   arrow_array_reader/BinaryViewArray/dictionary encoded, optional, half NULLs                                1.00    251.7±1.58µs        ? ?/sec    1.04    260.6±2.16µs        ? ?/sec
   arrow_array_reader/BinaryViewArray/dictionary encoded, optional, no NULLs                                  1.00    254.4±3.04µs        ? ?/sec    1.04    265.8±3.68µs        ? ?/sec
   arrow_array_reader/BinaryViewArray/plain encoded, mandatory, no NULLs                                      1.00    292.1±6.66µs        ? ?/sec    1.02    299.3±3.87µs        ? ?/sec
   arrow_array_reader/BinaryViewArray/plain encoded, mandatory, no NULLs, short string                        1.07    306.1±2.32µs        ? ?/sec    1.00    286.0±9.40µs        ? ?/sec
   arrow_array_reader/BinaryViewArray/plain encoded, optional, half NULLs                                     1.00    282.5±1.96µs        ? ?/sec    1.05    297.3±3.69µs        ? ?/sec
   arrow_array_reader/BinaryViewArray/plain encoded, optional, no NULLs                                       1.00    299.8±3.69µs        ? ?/sec    1.03   308.8±12.05µs        ? ?/sec
   arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Decimal128Array/byte_stream_split encoded, mandatory, no NULLs     1.04   1110.1±9.12µs        ? ?/sec    1.00   1068.3±8.01µs        ? ?/sec
   arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Decimal128Array/byte_stream_split encoded, optional, half NULLs    1.08    967.6±9.56µs        ? ?/sec    1.00    896.3±9.00µs        ? ?/sec
   arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Decimal128Array/byte_stream_split encoded, optional, no NULLs      1.04  1117.8±14.67µs        ? ?/sec    1.00   1073.8±5.56µs        ? ?/sec
   arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Decimal128Array/plain encoded, mandatory, no NULLs                 1.00    444.4±4.21µs        ? ?/sec    1.00    442.5±4.61µs        ? ?/sec
   arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Decimal128Array/plain encoded, optional, half NULLs                1.09    627.5±3.36µs        ? ?/sec    1.00   576.1±15.28µs        ? ?/sec
   arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Decimal128Array/plain encoded, optional, no NULLs                  1.01   454.3±10.47µs        ? ?/sec    1.00    449.0±3.72µs        ? ?/sec
   arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Float16Array/byte_stream_split encoded, mandatory, no NULLs        1.04    203.2±3.16µs        ? ?/sec    1.00    195.3±1.84µs        ? ?/sec
   arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Float16Array/byte_stream_split encoded, optional, half NULLs       1.00    317.6±5.04µs        ? ?/sec    1.05    334.8±3.89µs        ? ?/sec
   arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Float16Array/byte_stream_split encoded, optional, no NULLs         1.04    207.9±2.49µs        ? ?/sec    1.00    200.3±4.22µs        ? ?/sec
   arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Float16Array/plain encoded, mandatory, no NULLs                    1.01    118.3±4.67µs        ? ?/sec    1.00    117.5±0.50µs        ? ?/sec
   arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Float16Array/plain encoded, optional, half NULLs                   1.00    273.6±6.50µs        ? ?/sec    1.08    296.5±5.88µs        ? ?/sec
   arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Float16Array/plain encoded, optional, no NULLs                     1.00    122.5±0.72µs        ? ?/sec    1.00    122.4±0.60µs        ? ?/sec
   arrow_array_reader/FixedLenByteArray(16)/byte_stream_split encoded, mandatory, no NULLs                    1.06    732.5±9.92µs        ? ?/sec    1.00    688.5±5.53µs        ? ?/sec
   arrow_array_reader/FixedLenByteArray(16)/byte_stream_split encoded, optional, half NULLs                   1.15    588.8±3.06µs        ? ?/sec    1.00    510.9±2.93µs        ? ?/sec
   arrow_array_reader/FixedLenByteArray(16)/byte_stream_split encoded, optional, no NULLs                     1.07    741.3±7.43µs        ? ?/sec    1.00    693.6±8.30µs        ? ?/sec
   arrow_array_reader/FixedLenByteArray(16)/plain encoded, mandatory, no NULLs                                1.04     63.5±2.32µs        ? ?/sec    1.00     61.0±6.08µs        ? ?/sec
   arrow_array_reader/FixedLenByteArray(16)/plain encoded, optional, half NULLs                               1.25    258.2±3.99µs        ? ?/sec    1.00    206.3±2.86µs        ? ?/sec
   arrow_array_reader/FixedLenByteArray(16)/plain encoded, optional, no NULLs                                 1.00     68.6±1.99µs        ? ?/sec    1.01     69.7±7.70µs        ? ?/sec
   arrow_array_reader/FixedLenByteArray(2)/byte_stream_split encoded, mandatory, no NULLs                     1.09     93.9±0.48µs        ? ?/sec    1.00     85.9±0.68µs        ? ?/sec
   arrow_array_reader/FixedLenByteArray(2)/byte_stream_split encoded, optional, half NULLs                    1.00    209.5±2.00µs        ? ?/sec    1.08    226.4±1.30µs        ? ?/sec
   arrow_array_reader/FixedLenByteArray(2)/byte_stream_split encoded, optional, no NULLs                      1.09     99.5±0.57µs        ? ?/sec    1.00     91.1±0.86µs        ? ?/sec
   arrow_array_reader/FixedLenByteArray(2)/plain encoded, mandatory, no NULLs                                 1.00      9.2±0.27µs        ? ?/sec    1.01      9.3±0.18µs        ? ?/sec
   arrow_array_reader/FixedLenByteArray(2)/plain encoded, optional, half NULLs                                1.00    167.9±3.89µs        ? ?/sec    1.12    188.5±1.15µs        ? ?/sec
   arrow_array_reader/FixedLenByteArray(2)/plain encoded, optional, no NULLs                                  1.00     14.0±0.15µs        ? ?/sec    1.03     14.5±0.21µs        ? ?/sec
   arrow_array_reader/FixedLenByteArray(4)/byte_stream_split encoded, mandatory, no NULLs                     1.08    184.8±4.72µs        ? ?/sec    1.00    170.6±2.79µs        ? ?/sec
   arrow_array_reader/FixedLenByteArray(4)/byte_stream_split encoded, optional, half NULLs                    1.00    332.8±1.81µs        ? ?/sec    1.00    334.0±1.52µs        ? ?/sec
   arrow_array_reader/FixedLenByteArray(4)/byte_stream_split encoded, optional, no NULLs                      1.08    190.2±2.16µs        ? ?/sec    1.00    176.3±3.31µs        ? ?/sec
   arrow_array_reader/FixedLenByteArray(4)/plain encoded, mandatory, no NULLs                                 1.01     14.7±0.39µs        ? ?/sec    1.00     14.6±0.51µs        ? ?/sec
   arrow_array_reader/FixedLenByteArray(4)/plain encoded, optional, half NULLs                                1.00    246.2±1.55µs        ? ?/sec    1.05    257.3±1.99µs        ? ?/sec
   arrow_array_reader/FixedLenByteArray(4)/plain encoded, optional, no NULLs                                  1.00     19.6±0.38µs        ? ?/sec    1.01     19.8±0.40µs        ? ?/sec
   arrow_array_reader/FixedLenByteArray(8)/byte_stream_split encoded, mandatory, no NULLs                     1.07    365.4±2.38µs        ? ?/sec    1.00    342.1±2.69µs        ? ?/sec
   arrow_array_reader/FixedLenByteArray(8)/byte_stream_split encoded, optional, half NULLs                    1.20   396.1±13.52µs        ? ?/sec    1.00   330.6±11.05µs        ? ?/sec
   arrow_array_reader/FixedLenByteArray(8)/byte_stream_split encoded, optional, no NULLs                      1.08   377.5±43.51µs        ? ?/sec    1.00    349.1±1.59µs        ? ?/sec
   arrow_array_reader/FixedLenByteArray(8)/plain encoded, mandatory, no NULLs                                 1.10     25.4±0.44µs        ? ?/sec    1.00     23.0±1.30µs        ? ?/sec
   arrow_array_reader/FixedLenByteArray(8)/plain encoded, optional, half NULLs                                1.29    223.5±4.35µs        ? ?/sec    1.00    173.0±4.86µs        ? ?/sec
   arrow_array_reader/FixedLenByteArray(8)/plain encoded, optional, no NULLs                                  1.00     32.0±0.66µs        ? ?/sec    1.04     33.3±0.46µs        ? ?/sec
   arrow_array_reader/INT32/Decimal128Array/binary packed skip, mandatory, no NULLs                           1.00    111.1±2.48µs        ? ?/sec    1.00    111.1±3.53µs        ? ?/sec
   arrow_array_reader/INT32/Decimal128Array/binary packed skip, optional, half NULLs                          1.00    130.6±1.68µs        ? ?/sec    1.00    130.7±0.50µs        ? ?/sec
   arrow_array_reader/INT32/Decimal128Array/binary packed skip, optional, no NULLs                            1.01    115.5±0.80µs        ? ?/sec    1.00    114.3±1.69µs        ? ?/sec
   arrow_array_reader/INT32/Decimal128Array/binary packed, mandatory, no NULLs                                1.00    161.8±1.72µs        ? ?/sec    1.00    161.9±0.67µs        ? ?/sec
   arrow_array_reader/INT32/Decimal128Array/binary packed, optional, half NULLs                               1.01    225.2±1.15µs        ? ?/sec    1.00    223.3±1.71µs        ? ?/sec
   arrow_array_reader/INT32/Decimal128Array/binary packed, optional, no NULLs                                 1.00    167.2±2.45µs        ? ?/sec    1.00    167.0±0.90µs        ? ?/sec
   arrow_array_reader/INT32/Decimal128Array/byte_stream_split encoded, mandatory, no NULLs                    1.00     76.8±0.35µs        ? ?/sec    1.01     77.7±0.67µs        ? ?/sec
   arrow_array_reader/INT32/Decimal128Array/byte_stream_split encoded, optional, half NULLs                   1.02    181.7±1.82µs        ? ?/sec    1.00    178.2±1.43µs        ? ?/sec
   arrow_array_reader/INT32/Decimal128Array/byte_stream_split encoded, optional, no NULLs                     1.00     82.2±0.38µs        ? ?/sec    1.01     83.3±0.45µs        ? ?/sec
   arrow_array_reader/INT32/Decimal128Array/dictionary encoded, mandatory, no NULLs                           1.00    144.2±3.03µs        ? ?/sec    1.01    145.7±1.91µs        ? ?/sec
   arrow_array_reader/INT32/Decimal128Array/dictionary encoded, optional, half NULLs                          1.00    216.7±1.06µs        ? ?/sec    1.00    216.6±1.68µs        ? ?/sec
   arrow_array_reader/INT32/Decimal128Array/dictionary encoded, optional, no NULLs                            1.00    149.2±2.07µs        ? ?/sec    1.01    151.2±1.96µs        ? ?/sec
   arrow_array_reader/INT32/Decimal128Array/plain encoded, mandatory, no NULLs                                1.00     72.0±0.76µs        ? ?/sec    1.04     75.2±0.96µs        ? ?/sec
   arrow_array_reader/INT32/Decimal128Array/plain encoded, optional, half NULLs                               1.00    178.2±1.42µs        ? ?/sec    1.00    178.0±1.17µs        ? ?/sec
   arrow_array_reader/INT32/Decimal128Array/plain encoded, optional, no NULLs                                 1.00     79.8±0.38µs        ? ?/sec    1.01     80.6±0.42µs        ? ?/sec
   arrow_array_reader/INT64/Decimal128Array/binary packed skip, mandatory, no NULLs                           1.00    109.9±1.71µs        ? ?/sec    1.01    110.5±6.84µs        ? ?/sec
   arrow_array_reader/INT64/Decimal128Array/binary packed skip, optional, half NULLs                          1.02    130.3±3.90µs        ? ?/sec    1.00    128.2±0.95µs        ? ?/sec
   arrow_array_reader/INT64/Decimal128Array/binary packed skip, optional, no NULLs                            1.00    111.0±1.23µs        ? ?/sec    1.00    110.8±0.41µs        ? ?/sec
   arrow_array_reader/INT64/Decimal128Array/binary packed, mandatory, no NULLs                                1.01    163.9±4.84µs        ? ?/sec    1.00    162.5±0.77µs        ? ?/sec
   arrow_array_reader/INT64/Decimal128Array/binary packed, optional, half NULLs                               1.01    230.7±2.93µs        ? ?/sec    1.00    227.7±1.38µs        ? ?/sec
   arrow_array_reader/INT64/Decimal128Array/binary packed, optional, no NULLs                                 1.00    168.3±1.76µs        ? ?/sec    1.00    167.7±0.71µs        ? ?/sec
   arrow_array_reader/INT64/Decimal128Array/byte_stream_split encoded, mandatory, no NULLs                    1.00    201.5±1.83µs        ? ?/sec    1.01    202.7±2.03µs        ? ?/sec
   arrow_array_reader/INT64/Decimal128Array/byte_stream_split encoded, optional, half NULLs                   1.01    247.3±2.82µs        ? ?/sec    1.00    245.2±3.75µs        ? ?/sec
   arrow_array_reader/INT64/Decimal128Array/byte_stream_split encoded, optional, no NULLs                     1.00    207.4±0.52µs        ? ?/sec    1.01    208.9±2.75µs        ? ?/sec
   arrow_array_reader/INT64/Decimal128Array/dictionary encoded, mandatory, no NULLs                           1.00    142.3±0.88µs        ? ?/sec    1.00    142.6±0.52µs        ? ?/sec
   arrow_array_reader/INT64/Decimal128Array/dictionary encoded, optional, half NULLs                          1.00    214.4±1.54µs        ? ?/sec    1.00    213.4±1.44µs        ? ?/sec
   arrow_array_reader/INT64/Decimal128Array/dictionary encoded, optional, no NULLs                            1.00    148.2±1.15µs        ? ?/sec    1.00    148.8±2.45µs        ? ?/sec
   arrow_array_reader/INT64/Decimal128Array/plain encoded, mandatory, no NULLs                                1.00    103.7±1.54µs        ? ?/sec    1.02    105.8±1.30µs        ? ?/sec
   arrow_array_reader/INT64/Decimal128Array/plain encoded, optional, half NULLs                               1.04    197.7±3.76µs        ? ?/sec    1.00    190.9±3.35µs        ? ?/sec
   arrow_array_reader/INT64/Decimal128Array/plain encoded, optional, no NULLs                                 1.00    108.8±1.67µs        ? ?/sec    1.02    111.5±0.89µs        ? ?/sec
   arrow_array_reader/Int16Array/binary packed skip, mandatory, no NULLs                                      1.00     78.3±1.17µs        ? ?/sec    1.00     78.0±0.62µs        ? ?/sec
   arrow_array_reader/Int16Array/binary packed skip, optional, half NULLs                                     1.00    103.1±0.68µs        ? ?/sec    1.01    104.0±0.53µs        ? ?/sec
   arrow_array_reader/Int16Array/binary packed skip, optional, no NULLs                                       1.00     80.7±0.39µs        ? ?/sec    1.00     80.9±2.64µs        ? ?/sec
   arrow_array_reader/Int16Array/binary packed, mandatory, no NULLs                                           1.00    108.7±1.09µs        ? ?/sec    1.00    108.4±0.72µs        ? ?/sec
   arrow_array_reader/Int16Array/binary packed, optional, half NULLs                                          1.00    175.9±6.62µs        ? ?/sec    1.00    175.5±0.77µs        ? ?/sec
   arrow_array_reader/Int16Array/binary packed, optional, no NULLs                                            1.00    112.9±0.49µs        ? ?/sec    1.00    112.7±1.13µs        ? ?/sec
   arrow_array_reader/Int16Array/byte_stream_split encoded, mandatory, no NULLs                               1.00     42.3±0.72µs        ? ?/sec    1.04     43.8±0.25µs        ? ?/sec
   arrow_array_reader/Int16Array/byte_stream_split encoded, optional, half NULLs                              1.00    140.1±2.82µs        ? ?/sec    1.01    141.8±2.72µs        ? ?/sec
   arrow_array_reader/Int16Array/byte_stream_split encoded, optional, no NULLs                                1.00     46.3±0.22µs        ? ?/sec    1.04     48.0±0.19µs        ? ?/sec
   arrow_array_reader/Int16Array/dictionary encoded, mandatory, no NULLs                                      1.00    110.6±4.55µs        ? ?/sec    1.00    110.8±0.45µs        ? ?/sec
   arrow_array_reader/Int16Array/dictionary encoded, optional, half NULLs                                     1.00    178.9±5.36µs        ? ?/sec    1.01    180.7±6.86µs        ? ?/sec
   arrow_array_reader/Int16Array/dictionary encoded, optional, no NULLs                                       1.00    114.9±1.13µs        ? ?/sec    1.01    115.6±1.11µs        ? ?/sec
   arrow_array_reader/Int16Array/plain encoded, mandatory, no NULLs                                           1.01     38.3±0.56µs        ? ?/sec    1.00     37.7±0.35µs        ? ?/sec
   arrow_array_reader/Int16Array/plain encoded, optional, half NULLs                                          1.00    138.3±1.13µs        ? ?/sec    1.01    139.9±4.51µs        ? ?/sec
   arrow_array_reader/Int16Array/plain encoded, optional, no NULLs                                            1.00     42.3±0.21µs        ? ?/sec    1.00     42.5±0.63µs        ? ?/sec
   arrow_array_reader/Int32Array/binary packed skip, mandatory, no NULLs                                      1.00     84.4±0.45µs        ? ?/sec    1.00     84.3±2.11µs        ? ?/sec
   arrow_array_reader/Int32Array/binary packed skip, optional, half NULLs                                     1.01    103.7±1.22µs        ? ?/sec    1.00    103.0±0.64µs        ? ?/sec
   arrow_array_reader/Int32Array/binary packed skip, optional, no NULLs                                       1.01     87.5±1.75µs        ? ?/sec    1.00     86.9±0.61µs        ? ?/sec
   arrow_array_reader/Int32Array/binary packed, mandatory, no NULLs                                           1.00    110.3±0.73µs        ? ?/sec    1.00    109.9±1.23µs        ? ?/sec
   arrow_array_reader/Int32Array/binary packed, optional, half NULLs                                          1.00    167.8±1.33µs        ? ?/sec    1.01    168.9±3.22µs        ? ?/sec
   arrow_array_reader/Int32Array/binary packed, optional, no NULLs                                            1.00    114.5±0.89µs        ? ?/sec    1.00    114.3±1.43µs        ? ?/sec
   arrow_array_reader/Int32Array/byte_stream_split encoded, mandatory, no NULLs                               1.01     26.1±0.36µs        ? ?/sec    1.00     25.8±0.28µs        ? ?/sec
   arrow_array_reader/Int32Array/byte_stream_split encoded, optional, half NULLs                              1.00    123.3±1.05µs        ? ?/sec    1.01    125.1±1.12µs        ? ?/sec
   arrow_array_reader/Int32Array/byte_stream_split encoded, optional, no NULLs                                1.00     30.5±0.51µs        ? ?/sec    1.00     30.4±0.35µs        ? ?/sec
   arrow_array_reader/Int32Array/dictionary encoded, mandatory, no NULLs                                      1.00     92.9±1.04µs        ? ?/sec    1.00     93.2±0.72µs        ? ?/sec
   arrow_array_reader/Int32Array/dictionary encoded, optional, half NULLs                                     1.00    161.0±1.80µs        ? ?/sec    1.01    161.9±0.71µs        ? ?/sec
   arrow_array_reader/Int32Array/dictionary encoded, optional, no NULLs                                       1.00     97.7±0.75µs        ? ?/sec    1.00     98.2±0.81µs        ? ?/sec
   arrow_array_reader/Int32Array/plain encoded, mandatory, no NULLs                                           1.01     18.1±0.85µs        ? ?/sec    1.00     17.9±0.51µs        ? ?/sec
   arrow_array_reader/Int32Array/plain encoded, optional, half NULLs                                          1.00    120.8±2.02µs        ? ?/sec    1.01    121.8±1.07µs        ? ?/sec
   arrow_array_reader/Int32Array/plain encoded, optional, no NULLs                                            1.00     23.9±0.54µs        ? ?/sec    1.01     24.1±0.44µs        ? ?/sec
   arrow_array_reader/Int64Array/binary packed skip, mandatory, no NULLs                                      1.00     81.6±0.70µs        ? ?/sec    1.00     81.6±0.40µs        ? ?/sec
   arrow_array_reader/Int64Array/binary packed skip, optional, half NULLs                                     1.00    101.2±0.96µs        ? ?/sec    1.00    101.5±0.69µs        ? ?/sec
   arrow_array_reader/Int64Array/binary packed skip, optional, no NULLs                                       1.00     84.2±1.38µs        ? ?/sec    1.00     83.9±0.39µs        ? ?/sec
   arrow_array_reader/Int64Array/binary packed, mandatory, no NULLs                                           1.03    113.4±7.62µs        ? ?/sec    1.00    110.3±1.48µs        ? ?/sec
   arrow_array_reader/Int64Array/binary packed, optional, half NULLs                                          1.00    166.7±2.56µs        ? ?/sec    1.00    167.2±2.56µs        ? ?/sec
   arrow_array_reader/Int64Array/binary packed, optional, no NULLs                                            1.00    114.8±1.14µs        ? ?/sec    1.00    114.9±0.75µs        ? ?/sec
   arrow_array_reader/Int64Array/byte_stream_split encoded, mandatory, no NULLs                               1.00    148.5±0.45µs        ? ?/sec    1.01    149.4±0.80µs        ? ?/sec
   arrow_array_reader/Int64Array/byte_stream_split encoded, optional, half NULLs                              1.00    190.3±3.41µs        ? ?/sec    1.00    190.2±5.21µs        ? ?/sec
   arrow_array_reader/Int64Array/byte_stream_split encoded, optional, no NULLs                                1.00    153.2±0.51µs        ? ?/sec    1.01    154.3±0.59µs        ? ?/sec
   arrow_array_reader/Int64Array/dictionary encoded, mandatory, no NULLs                                      1.00     90.6±5.69µs        ? ?/sec    1.01     91.3±0.77µs        ? ?/sec
   arrow_array_reader/Int64Array/dictionary encoded, optional, half NULLs                                     1.00    156.8±2.18µs        ? ?/sec    1.01    158.5±1.23µs        ? ?/sec
   arrow_array_reader/Int64Array/dictionary encoded, optional, no NULLs                                       1.00     94.0±1.20µs        ? ?/sec    1.02     95.7±1.44µs        ? ?/sec
   arrow_array_reader/Int64Array/plain encoded, mandatory, no NULLs                                           1.18     50.6±3.12µs        ? ?/sec    1.00     42.9±2.31µs        ? ?/sec
   arrow_array_reader/Int64Array/plain encoded, optional, half NULLs                                          1.01    133.8±0.85µs        ? ?/sec    1.00    133.0±1.02µs        ? ?/sec
   arrow_array_reader/Int64Array/plain encoded, optional, no NULLs                                            1.15     56.8±3.06µs        ? ?/sec    1.00     49.5±2.11µs        ? ?/sec
   arrow_array_reader/Int8Array/binary packed skip, mandatory, no NULLs                                       1.01     82.7±1.47µs        ? ?/sec    1.00     82.2±0.63µs        ? ?/sec
   arrow_array_reader/Int8Array/binary packed skip, optional, half NULLs                                      1.00    103.7±1.12µs        ? ?/sec    1.00    104.1±1.41µs        ? ?/sec
   arrow_array_reader/Int8Array/binary packed skip, optional, no NULLs                                        1.01     85.3±0.95µs        ? ?/sec    1.00     84.8±0.32µs        ? ?/sec
   arrow_array_reader/Int8Array/binary packed, mandatory, no NULLs                                            1.01    110.9±0.55µs        ? ?/sec    1.00    110.0±0.68µs        ? ?/sec
   arrow_array_reader/Int8Array/binary packed, optional, half NULLs                                           1.00    172.2±3.18µs        ? ?/sec    1.01    173.2±3.39µs        ? ?/sec
   arrow_array_reader/Int8Array/binary packed, optional, no NULLs                                             1.00    115.1±0.49µs        ? ?/sec    1.00    114.6±0.74µs        ? ?/sec
   arrow_array_reader/Int8Array/byte_stream_split encoded, mandatory, no NULLs                                1.00     36.0±0.35µs        ? ?/sec    1.00     35.8±0.37µs        ? ?/sec
   arrow_array_reader/Int8Array/byte_stream_split encoded, optional, half NULLs                               1.00    134.1±0.64µs        ? ?/sec    1.00    134.3±1.32µs        ? ?/sec
   arrow_array_reader/Int8Array/byte_stream_split encoded, optional, no NULLs                                 1.00     40.2±0.20µs        ? ?/sec    1.00     40.1±0.20µs        ? ?/sec
   arrow_array_reader/Int8Array/dictionary encoded, mandatory, no NULLs                                       1.00    102.7±1.06µs        ? ?/sec    1.00    102.9±1.40µs        ? ?/sec
   arrow_array_reader/Int8Array/dictionary encoded, optional, half NULLs                                      1.00    171.4±1.60µs        ? ?/sec    1.01    172.5±4.54µs        ? ?/sec
   arrow_array_reader/Int8Array/dictionary encoded, optional, no NULLs                                        1.00    107.4±0.97µs        ? ?/sec    1.00    107.5±1.49µs        ? ?/sec
   arrow_array_reader/Int8Array/plain encoded, mandatory, no NULLs                                            1.01     30.3±0.19µs        ? ?/sec    1.00     30.0±0.73µs        ? ?/sec
   arrow_array_reader/Int8Array/plain encoded, optional, half NULLs                                           1.00    132.0±8.03µs        ? ?/sec    1.00    131.4±0.71µs        ? ?/sec
   arrow_array_reader/Int8Array/plain encoded, optional, no NULLs                                             1.01     34.8±0.30µs        ? ?/sec    1.00     34.5±0.23µs        ? ?/sec
   arrow_array_reader/ListArray/plain encoded optional strings half NULLs                                     1.02      7.4±0.11ms        ? ?/sec    1.00      7.2±0.07ms        ? ?/sec
   arrow_array_reader/ListArray/plain encoded optional strings no NULLs                                       1.02     13.1±0.13ms        ? ?/sec    1.00     12.8±0.18ms        ? ?/sec
   arrow_array_reader/StringArray/dictionary encoded, mandatory, no NULLs                                     1.00    490.6±8.12µs        ? ?/sec    1.06    519.0±4.69µs        ? ?/sec
   arrow_array_reader/StringArray/dictionary encoded, optional, half NULLs                                    1.00   662.4±11.83µs        ? ?/sec    1.05    697.3±7.51µs        ? ?/sec
   arrow_array_reader/StringArray/dictionary encoded, optional, no NULLs                                      1.00    487.2±5.59µs        ? ?/sec    1.07    522.4±8.19µs        ? ?/sec
   arrow_array_reader/StringArray/plain encoded, mandatory, no NULLs                                          1.15   752.7±12.61µs        ? ?/sec    1.00    655.6±5.39µs        ? ?/sec
   arrow_array_reader/StringArray/plain encoded, optional, half NULLs                                         1.03    825.6±6.19µs        ? ?/sec    1.00    801.0±7.88µs        ? ?/sec
   arrow_array_reader/StringArray/plain encoded, optional, no NULLs                                           1.14    759.4±9.77µs        ? ?/sec    1.00    664.0±3.92µs        ? ?/sec
   arrow_array_reader/StringDictionary/dictionary encoded, mandatory, no NULLs                                1.00   335.4±13.73µs        ? ?/sec    1.01    337.9±9.18µs        ? ?/sec
   arrow_array_reader/StringDictionary/dictionary encoded, optional, half NULLs                               1.00    410.1±6.51µs        ? ?/sec    1.01    414.3±8.04µs        ? ?/sec
   arrow_array_reader/StringDictionary/dictionary encoded, optional, no NULLs                                 1.00    338.1±4.19µs        ? ?/sec    1.02    343.6±6.67µs        ? ?/sec
   arrow_array_reader/StringViewArray/dictionary encoded, mandatory, no NULLs                                 1.00    248.8±5.37µs        ? ?/sec    1.04    259.1±3.10µs        ? ?/sec
   arrow_array_reader/StringViewArray/dictionary encoded, optional, half NULLs                                1.00    252.5±1.48µs        ? ?/sec    1.00    253.3±4.29µs        ? ?/sec
   arrow_array_reader/StringViewArray/dictionary encoded, optional, no NULLs                                  1.00    256.3±4.60µs        ? ?/sec    1.00    257.4±3.37µs        ? ?/sec
   arrow_array_reader/StringViewArray/plain encoded, mandatory, no NULLs                                      1.00   455.4±10.29µs        ? ?/sec    1.03   466.8±29.76µs        ? ?/sec
   arrow_array_reader/StringViewArray/plain encoded, optional, half NULLs                                     1.00    357.1±2.53µs        ? ?/sec    1.05    375.4±6.64µs        ? ?/sec
   arrow_array_reader/StringViewArray/plain encoded, optional, no NULLs                                       1.00    460.8±3.19µs        ? ?/sec    1.02   471.0±11.44µs        ? ?/sec
   arrow_array_reader/UInt16Array/binary packed skip, mandatory, no NULLs                                     1.00     94.7±0.72µs        ? ?/sec    1.00     94.5±1.32µs        ? ?/sec
   arrow_array_reader/UInt16Array/binary packed skip, optional, half NULLs                                    1.00    112.2±1.10µs        ? ?/sec    1.00    112.7±2.40µs        ? ?/sec
   arrow_array_reader/UInt16Array/binary packed skip, optional, no NULLs                                      1.00     96.6±0.66µs        ? ?/sec    1.00     96.4±1.40µs        ? ?/sec
   arrow_array_reader/UInt16Array/binary packed, mandatory, no NULLs                                          1.01    129.6±7.49µs        ? ?/sec    1.00    128.2±0.54µs        ? ?/sec
   arrow_array_reader/UInt16Array/binary packed, optional, half NULLs                                         1.00    187.9±2.48µs        ? ?/sec    1.00    187.7±0.98µs        ? ?/sec
   arrow_array_reader/UInt16Array/binary packed, optional, no NULLs                                           1.00    133.0±1.63µs        ? ?/sec    1.00    132.9±1.05µs        ? ?/sec
   arrow_array_reader/UInt16Array/byte_stream_split encoded, mandatory, no NULLs                              1.00     42.1±0.20µs        ? ?/sec    1.00     42.2±0.49µs        ? ?/sec
   arrow_array_reader/UInt16Array/byte_stream_split encoded, optional, half NULLs                             1.00    141.5±1.19µs        ? ?/sec    1.00    141.4±0.57µs        ? ?/sec
   arrow_array_reader/UInt16Array/byte_stream_split encoded, optional, no NULLs                               1.00     46.4±0.33µs        ? ?/sec    1.01     46.7±0.47µs        ? ?/sec
   arrow_array_reader/UInt16Array/dictionary encoded, mandatory, no NULLs                                     1.00    110.6±1.14µs        ? ?/sec    1.00    110.4±0.36µs        ? ?/sec
   arrow_array_reader/UInt16Array/dictionary encoded, optional, half NULLs                                    1.00    179.6±1.54µs        ? ?/sec    1.00    180.1±2.71µs        ? ?/sec
   arrow_array_reader/UInt16Array/dictionary encoded, optional, no NULLs                                      1.00    115.2±0.41µs        ? ?/sec    1.00    115.4±0.64µs        ? ?/sec
   arrow_array_reader/UInt16Array/plain encoded, mandatory, no NULLs                                          1.00     38.1±0.22µs        ? ?/sec    1.00     38.1±0.32µs        ? ?/sec
   arrow_array_reader/UInt16Array/plain encoded, optional, half NULLs                                         1.00    139.6±1.13µs        ? ?/sec    1.00    139.7±2.41µs        ? ?/sec
   arrow_array_reader/UInt16Array/plain encoded, optional, no NULLs                                           1.00     42.6±0.25µs        ? ?/sec    1.00     42.7±0.55µs        ? ?/sec
   arrow_array_reader/UInt32Array/binary packed skip, mandatory, no NULLs                                     1.00     84.6±1.39µs        ? ?/sec    1.00     84.5±1.79µs        ? ?/sec
   arrow_array_reader/UInt32Array/binary packed skip, optional, half NULLs                                    1.00    102.4±0.60µs        ? ?/sec    1.00    102.9±0.81µs        ? ?/sec
   arrow_array_reader/UInt32Array/binary packed skip, optional, no NULLs                                      1.00     87.0±1.10µs        ? ?/sec    1.00     87.1±2.13µs        ? ?/sec
   arrow_array_reader/UInt32Array/binary packed, mandatory, no NULLs                                          1.00    111.0±4.37µs        ? ?/sec    1.00    111.3±1.35µs        ? ?/sec
   arrow_array_reader/UInt32Array/binary packed, optional, half NULLs                                         1.00    167.5±2.89µs        ? ?/sec    1.01    168.5±3.24µs        ? ?/sec
   arrow_array_reader/UInt32Array/binary packed, optional, no NULLs                                           1.00    114.5±0.90µs        ? ?/sec    1.00    114.3±1.43µs        ? ?/sec
   arrow_array_reader/UInt32Array/byte_stream_split encoded, mandatory, no NULLs                              1.00     26.3±0.33µs        ? ?/sec    1.00     26.2±0.35µs        ? ?/sec
   arrow_array_reader/UInt32Array/byte_stream_split encoded, optional, half NULLs                             1.00    122.8±0.61µs        ? ?/sec    1.01    123.7±0.49µs        ? ?/sec
   arrow_array_reader/UInt32Array/byte_stream_split encoded, optional, no NULLs                               1.00     30.1±0.32µs        ? ?/sec    1.01     30.4±0.33µs        ? ?/sec
   arrow_array_reader/UInt32Array/dictionary encoded, mandatory, no NULLs                                     1.00     93.5±1.37µs        ? ?/sec    1.00     93.6±0.95µs        ? ?/sec
   arrow_array_reader/UInt32Array/dictionary encoded, optional, half NULLs                                    1.00    160.8±1.20µs        ? ?/sec    1.01    162.3±3.15µs        ? ?/sec
   arrow_array_reader/UInt32Array/dictionary encoded, optional, no NULLs                                      1.00     98.0±0.53µs        ? ?/sec    1.00     98.1±1.15µs        ? ?/sec
   arrow_array_reader/UInt32Array/plain encoded, mandatory, no NULLs                                          1.01     21.1±0.80µs        ? ?/sec    1.00     20.9±0.56µs        ? ?/sec
   arrow_array_reader/UInt32Array/plain encoded, optional, half NULLs                                         1.00    120.4±1.90µs        ? ?/sec    1.01    122.2±1.19µs        ? ?/sec
   arrow_array_reader/UInt32Array/plain encoded, optional, no NULLs                                           1.03     26.0±0.87µs        ? ?/sec    1.00     25.3±0.52µs        ? ?/sec
   arrow_array_reader/UInt64Array/binary packed skip, mandatory, no NULLs                                     1.00     81.6±0.40µs        ? ?/sec    1.00     81.8±0.68µs        ? ?/sec
   arrow_array_reader/UInt64Array/binary packed skip, optional, half NULLs                                    1.00    101.6±0.63µs        ? ?/sec    1.00    101.8±1.32µs        ? ?/sec
   arrow_array_reader/UInt64Array/binary packed skip, optional, no NULLs                                      1.00     84.6±3.10µs        ? ?/sec    1.00     84.4±2.24µs        ? ?/sec
   arrow_array_reader/UInt64Array/binary packed, mandatory, no NULLs                                          1.01    111.3±1.54µs        ? ?/sec    1.00    110.3±0.65µs        ? ?/sec
   arrow_array_reader/UInt64Array/binary packed, optional, half NULLs                                         1.00    167.5±0.79µs        ? ?/sec    1.00    167.4±1.37µs        ? ?/sec
   arrow_array_reader/UInt64Array/binary packed, optional, no NULLs                                           1.00    114.1±1.13µs        ? ?/sec    1.00    114.5±1.13µs        ? ?/sec
   arrow_array_reader/UInt64Array/byte_stream_split encoded, mandatory, no NULLs                              1.00    148.7±0.57µs        ? ?/sec    1.01    149.6±1.08µs        ? ?/sec
   arrow_array_reader/UInt64Array/byte_stream_split encoded, optional, half NULLs                             1.00    190.0±3.48µs        ? ?/sec    1.01    192.0±5.92µs        ? ?/sec
   arrow_array_reader/UInt64Array/byte_stream_split encoded, optional, no NULLs                               1.00    153.2±0.97µs        ? ?/sec    1.01    154.3±0.56µs        ? ?/sec
   arrow_array_reader/UInt64Array/dictionary encoded, mandatory, no NULLs                                     1.00     90.2±0.65µs        ? ?/sec    1.00     90.1±0.81µs        ? ?/sec
   arrow_array_reader/UInt64Array/dictionary encoded, optional, half NULLs                                    1.00    156.9±3.84µs        ? ?/sec    1.01    158.0±1.08µs        ? ?/sec
   arrow_array_reader/UInt64Array/dictionary encoded, optional, no NULLs                                      1.00     95.3±4.99µs        ? ?/sec    1.00     95.0±0.66µs        ? ?/sec
   arrow_array_reader/UInt64Array/plain encoded, mandatory, no NULLs                                          1.12     48.5±2.95µs        ? ?/sec    1.00     43.2±2.48µs        ? ?/sec
   arrow_array_reader/UInt64Array/plain encoded, optional, half NULLs                                         1.02    135.4±4.55µs        ? ?/sec    1.00    133.2±2.27µs        ? ?/sec
   arrow_array_reader/UInt64Array/plain encoded, optional, no NULLs                                           1.06     52.1±2.60µs        ? ?/sec    1.00     49.4±2.46µs        ? ?/sec
   arrow_array_reader/UInt8Array/binary packed skip, mandatory, no NULLs                                      1.01     89.8±1.75µs        ? ?/sec    1.00     89.2±1.26µs        ? ?/sec
   arrow_array_reader/UInt8Array/binary packed skip, optional, half NULLs                                     1.00    107.1±1.10µs        ? ?/sec    1.01    107.9±0.43µs        ? ?/sec
   arrow_array_reader/UInt8Array/binary packed skip, optional, no NULLs                                       1.00     92.2±1.15µs        ? ?/sec    1.00     92.6±0.60µs        ? ?/sec
   arrow_array_reader/UInt8Array/binary packed, mandatory, no NULLs                                           1.00    120.4±2.42µs        ? ?/sec    1.00    119.9±2.17µs        ? ?/sec
   arrow_array_reader/UInt8Array/binary packed, optional, half NULLs                                          1.00    177.6±1.16µs        ? ?/sec    1.02   181.0±12.14µs        ? ?/sec
   arrow_array_reader/UInt8Array/binary packed, optional, no NULLs                                            1.00    124.5±1.62µs        ? ?/sec    1.00    124.1±0.37µs        ? ?/sec
   arrow_array_reader/UInt8Array/byte_stream_split encoded, mandatory, no NULLs                               1.00     35.8±0.36µs        ? ?/sec    1.00     36.0±0.45µs        ? ?/sec
   arrow_array_reader/UInt8Array/byte_stream_split encoded, optional, half NULLs                              1.00    132.5±1.84µs        ? ?/sec    1.01    134.0±1.10µs        ? ?/sec
   arrow_array_reader/UInt8Array/byte_stream_split encoded, optional, no NULLs                                1.00     40.1±0.44µs        ? ?/sec    1.00     40.1±0.25µs        ? ?/sec
   arrow_array_reader/UInt8Array/dictionary encoded, mandatory, no NULLs                                      1.00    102.8±1.29µs        ? ?/sec    1.00    103.0±0.65µs        ? ?/sec
   arrow_array_reader/UInt8Array/dictionary encoded, optional, half NULLs                                     1.00    170.9±1.25µs        ? ?/sec    1.01    171.8±2.15µs        ? ?/sec
   arrow_array_reader/UInt8Array/dictionary encoded, optional, no NULLs                                       1.00    107.4±1.74µs        ? ?/sec    1.01    107.9±1.12µs        ? ?/sec
   arrow_array_reader/UInt8Array/plain encoded, mandatory, no NULLs                                           1.01     30.2±0.51µs        ? ?/sec    1.00     29.9±0.25µs        ? ?/sec
   arrow_array_reader/UInt8Array/plain encoded, optional, half NULLs                                          1.00    131.0±2.59µs        ? ?/sec    1.01    132.1±0.68µs        ? ?/sec
   arrow_array_reader/UInt8Array/plain encoded, optional, no NULLs                                            1.02     35.3±0.29µs        ? ?/sec    1.00     34.7±0.46µs        ? ?/sec
   ```
   
   
   </p>
   </details>
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] WIP: Boolean mask backed RowSelection [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #8902:
URL: https://github.com/apache/arrow-rs/pull/8902#issuecomment-3647141874

   run benchmark arrow_reader arrow_reader_clickbench arrow_reader_row_filter


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] WIP: Boolean mask backed RowSelection [arrow-rs]

Posted by "rluvaton (via GitHub)" <gi...@apache.org>.
rluvaton commented on code in PR #8902:
URL: https://github.com/apache/arrow-rs/pull/8902#discussion_r2578756416


##########
parquet/src/arrow/arrow_reader/selection/and_then.rs:
##########
@@ -0,0 +1,347 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Optimized [`boolean_buffer_and_then`] kernel, based on code by
+//! Xiangpeng Hao in <https://github.com/apache/arrow-rs/pull/6624> and
+//! [LiquidCache's implementation](https://github.com/XiangpengHao/liquid-cache/blob/60323cdb5f17f88af633ad9acbc7d74f684f9912/src/parquet/src/utils.rs#L17)
+
+use crate::arrow::arrow_reader::RowSelector;
+use arrow_buffer::{BooleanBuffer, BooleanBufferBuilder, MutableBuffer};
+
+fn boolean_buffer_and_then_fallback(left: &BooleanBuffer, right: &BooleanBuffer) -> BooleanBuffer {
+    debug_assert_eq!(
+        left.count_set_bits(),
+        right.len(),
+        "the right selection must have the same number of set bits as the left selection"
+    );
+
+    if left.len() == right.len() {
+        debug_assert_eq!(left.count_set_bits(), left.len());
+        return right.clone();
+    }
+
+    let mut buffer = MutableBuffer::from_len_zeroed(left.values().len());
+    buffer.copy_from_slice(left.values());
+    let mut builder = BooleanBufferBuilder::new_from_buffer(buffer, left.len());
+
+    let mut other_bits = right.iter();
+
+    for bit_idx in left.set_indices() {
+        let predicate = other_bits
+            .next()
+            .expect("Mismatch in set bits between self and other");
+        if !predicate {
+            builder.set_bit(bit_idx, false);
+        }

Review Comment:
   maybe a faster approach would be to use set slices and then use AND with right starting from the last location



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] WIP: Boolean mask backed RowSelection [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #8902:
URL: https://github.com/apache/arrow-rs/pull/8902#issuecomment-4381515667

   I don't really have time to push this forward now


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org