You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/06/16 17:41:05 UTC

(datafusion) 01/01: refactor: use raw view access in do_append_val_inner and consolidate duplicated logic (#22907)

This is an automated email from the ASF dual-hosted git repository.

github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-22907-408dad3dc5733f1b438cf5515af3d8c0da4b178c
in repository https://gitbox.apache.org/repos/asf/datafusion.git

commit c7e92848f958065fec0f74669bbcb66ae067c2ad
Author: EeshanBembi <33...@users.noreply.github.com>
AuthorDate: Tue Jun 16 23:10:46 2026 +0530

    refactor: use raw view access in do_append_val_inner and consolidate duplicated logic (#22907)
    
    ## Which issue does this PR close?
    
    Follow-up to #21794, addressing review feedback from @alamb and
    @Dandandan.
    
    ## Rationale for this change
    
    In the review of #21794, several optimizations were suggested:
    
    - **@alamb**
    ([comment](https://github.com/apache/datafusion/pull/21794#discussion_r2174375498)):
    "as a possible future optimization, we could use `get_unchecked` here if
    it makes any difference" — referring to the slow-path `arr.views()[row]`
    access.
    - **@alamb**
    ([comment](https://github.com/apache/datafusion/pull/21794#discussion_r2174376693)):
    "from here on down I think this is basically the same as
    append_val_inner -- if there are any differences perhaps we can fold it
    into append_val_inner and avoid the copy"
    - **@Dandandan**
    ([comment](https://github.com/apache/datafusion/pull/21794#discussion_r2175204095)):
    "In principle we can make this faster as well - `extend` + reuse input
    view (instead of make_view) + avoid `array.value(row)`"
    
    ## What changes are included in this PR?
    
    **1. Refactored `do_append_val_inner` to use raw view access**
    
    Replaced `array.value(row)` + `make_view()` with raw view access via
    `get_unchecked(row)`:
    - **Inline (len <= 12):** push the u128 view as-is — no decode/re-encode
    round-trip
    - **Non-inline (len > 12):** parse via `ByteView::from(view)`, copy
    buffer data, reuse source prefix directly (avoids re-reading first 4
    bytes)
    
    **2. Simplified the vectorized slow path**
    
    Replaced the duplicated 28-line loop body in `vectorized_append_inner`
    with `try_reserve` + a loop calling `do_append_val_inner`, eliminating
    code duplication.
    
    **3. Removed unused `make_view` import**
    
    ### Safety notes
    
    - **`get_unchecked` usage**: Consistent with `do_equal_to_inner` (same
    file) and `PrimitiveGroupValueBuilder` in `primitive.rs`, both of which
    use the same pattern. All callers derive row indices from enumeration
    over the input array length, guaranteeing validity.
    - **Buffer access safety**: When `data_buffers()` is empty, all views
    must have len <= 12 (Arrow invariant), so the non-inline branch is never
    entered.
    
    ## Are these changes tested?
    
    Covered by 6 existing unit tests in the `bytes_view` module plus 3
    integration tests in the `multi_group_by` module. All 111 tests in the
    aggregates suite pass.
    
    ## Are there any user-facing changes?
    
    No. This is an internal refactor with no API changes.
---
 .../group_values/multi_group_by/bytes_view.rs      | 77 ++++++++--------------
 1 file changed, 29 insertions(+), 48 deletions(-)

diff --git a/datafusion/physical-plan/src/aggregates/group_values/multi_group_by/bytes_view.rs b/datafusion/physical-plan/src/aggregates/group_values/multi_group_by/bytes_view.rs
index abc3aba88a..8625772e2c 100644
--- a/datafusion/physical-plan/src/aggregates/group_values/multi_group_by/bytes_view.rs
+++ b/datafusion/physical-plan/src/aggregates/group_values/multi_group_by/bytes_view.rs
@@ -21,7 +21,6 @@ use crate::aggregates::group_values::multi_group_by::{
 use crate::aggregates::group_values::null_builder::MaybeNullBufferBuilder;
 use arrow::array::{
     Array, ArrayRef, AsArray, BooleanBufferBuilder, ByteView, GenericByteViewArray,
-    make_view,
 };
 use arrow::buffer::{Buffer, ScalarBuffer};
 use arrow::datatypes::ByteViewType;
@@ -176,9 +175,9 @@ impl<B: ByteViewType> ByteViewGroupValueBuilder<B> {
                     // copy them directly instead of going through value() → make_view().
                     self.views.extend(rows.iter().map(|&row| arr.views()[row]));
                 } else {
-                    // Slow path: some strings are non-inline (>12 bytes).
-                    // Read views directly to avoid array.value(row) overhead and
-                    // reuse the source view's prefix instead of recomputing it via make_view.
+                    // Slow path: some strings may be non-inline (>12 bytes).
+                    // Pre-reserve and delegate to do_append_val_inner which
+                    // reads raw views directly and reuses source prefixes.
                     self.views.try_reserve(rows.len()).map_err(|e| {
                         datafusion_common::exec_datafusion_err!(
                             "failed to reserve {0} views: {e}",
@@ -186,33 +185,7 @@ impl<B: ByteViewType> ByteViewGroupValueBuilder<B> {
                         )
                     })?;
                     for &row in rows {
-                        let view = arr.views()[row];
-                        let len = view as u32;
-                        if len <= 12 {
-                            // This row happens to be inline; copy view directly.
-                            self.views.push(view);
-                        } else {
-                            let src = ByteView::from(view);
-                            // ensure_in_progress_big_enough must be called before computing
-                            // new_buffer_index / new_offset — it may flush in_progress to completed.
-                            self.ensure_in_progress_big_enough(len as usize);
-                            let new_buffer_index = self.completed.len() as u32;
-                            let new_offset = self.in_progress.len() as u32;
-                            let src_buf = &arr.data_buffers()[src.buffer_index as usize];
-                            self.in_progress.extend_from_slice(
-                                &src_buf[src.offset as usize
-                                    ..(src.offset + src.length) as usize],
-                            );
-                            // Reuse prefix from the source view — avoids re-reading first 4 bytes.
-                            let new_view = ByteView {
-                                length: src.length,
-                                prefix: src.prefix,
-                                buffer_index: new_buffer_index,
-                                offset: new_offset,
-                            }
-                            .as_u128();
-                            self.views.push(new_view);
-                        }
+                        self.do_append_val_inner(arr, row);
                     }
                 }
             }
@@ -230,25 +203,33 @@ impl<B: ByteViewType> ByteViewGroupValueBuilder<B> {
     where
         B: ByteViewType,
     {
-        let value: &[u8] = array.value(row).as_ref();
+        // SAFETY: the caller ensures `row` is valid
+        let view = unsafe { *array.views().get_unchecked(row) };
+        let len = view as u32;
 
-        let value_len = value.len();
-        let view = if value_len <= 12 {
-            make_view(value, 0, 0)
+        if len <= 12 {
+            // Inline value: the view is already self-contained, push as-is.
+            self.views.push(view);
         } else {
-            // Ensure big enough block to hold the value firstly
-            self.ensure_in_progress_big_enough(value_len);
-
-            // Append value
-            let buffer_index = self.completed.len();
-            let offset = self.in_progress.len();
-            self.in_progress.extend_from_slice(value);
-
-            make_view(value, buffer_index as u32, offset as u32)
-        };
-
-        // Append view
-        self.views.push(view);
+            // Non-inline value: copy the buffer data and construct a new view
+            // that points into our own buffers, reusing the source prefix.
+            let src = ByteView::from(view);
+            self.ensure_in_progress_big_enough(len as usize);
+            let new_buffer_index = self.completed.len() as u32;
+            let new_offset = self.in_progress.len() as u32;
+            let src_buf = &array.data_buffers()[src.buffer_index as usize];
+            self.in_progress.extend_from_slice(
+                &src_buf[src.offset as usize..(src.offset + src.length) as usize],
+            );
+            let new_view = ByteView {
+                length: src.length,
+                prefix: src.prefix,
+                buffer_index: new_buffer_index,
+                offset: new_offset,
+            }
+            .as_u128();
+            self.views.push(new_view);
+        }
     }
 
     fn ensure_in_progress_big_enough(&mut self, value_len: usize) {


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org