You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "zhli1142015 (via GitHub)" <gi...@apache.org> on 2026/05/18 01:09:49 UTC

[PR] [VL] Enable Velox batch resizer copyRanges fast path [gluten]

zhli1142015 opened a new pull request, #12101:
URL: https://github.com/apache/gluten/pull/12101

   Add a default-enabled VeloxBatchResizer fast path that collects small dense batches, allocates the output RowVector once, and bulk-copies child vector ranges with copyRanges. The config remains available as an opt-out switch.
   
   Wire the flag through Scala, Java, and JNI, add C++ coverage for fast-path and fallback behavior, add config default coverage, and add dense-vector benchmark scenarios comparing the append opt-out baseline, default copyRanges path, direct child copyRanges, reader-side raw payload bulk-copy model, and pre-merged flush model.
   
   Benchmark results from velox_batch_resizer_benchmark (CPU time; ASLR enabled, so numbers may have noise):
   
   - Mixed_64x64: append opt-out baseline 95.1us, default copyRanges 19.7us, direct child copyRanges 17.4us, raw bulk-copy model 33.3us.
   
   - Mixed_16x256: append opt-out baseline 33.7us, default copyRanges 6.4us, direct child copyRanges 5.0us, raw bulk-copy model 10.5us.
   
   - Mixed_256x16: append opt-out baseline 217.7us, default copyRanges 50.4us, direct child copyRanges 28.6us, raw bulk-copy model 112.6us.
   
   - Fixed2_64x64: append opt-out baseline 26.6us, default copyRanges 5.5us, direct child copyRanges 2.0us, raw bulk-copy model 13.7us.
   
   - Fixed16_64x64: append opt-out baseline 121.6us, default copyRanges 27.0us, direct child copyRanges 17.4us, raw bulk-copy model 92.9us.
   
   - LongString_64x64: append opt-out baseline 31.7us, default copyRanges 7.1us, direct child copyRanges 4.5us, raw bulk-copy model 15.3us.
   
   - BoolHeavy_64x64: append opt-out baseline 68.7us, default copyRanges 10.9us, direct child copyRanges 5.4us, raw bulk-copy model 37.7us.
   
   <!--
   Thank you for submitting a pull request! Here are some tips:
   
   1. For first-time contributors, please read our contributing guide:
      https://github.com/apache/gluten/blob/main/CONTRIBUTING.md
   2. If necessary, create a GitHub issue for discussion beforehand to avoid duplicate work.
   3. If the PR is specific to a single backend, include [VL] or [CH] in the PR title to indicate the
      Velox or ClickHouse backend, respectively.
   4. If the PR is not ready for review, please mark it as a draft.
   -->
   
   ## What changes are proposed in this pull request?
   
   <!--
   Provide a clear and concise description of the changes introduced in this PR.
   Ensure the PR description aligns with the code changes, especially after updates.
   If applicable, include "Fixes #<GitHub_Issue_ID>" to automatically close the corresponding issue
   when the PR is merged.
   -->
   
   ## How was this patch tested?
   
   <!--
   Describe how the changes were tested, if applicable.
   Include new tests to validate the functionality, if necessary.
   For UI-related changes, attach screenshots to demonstrate the updates.
   -->
   
   ## Was this patch authored or co-authored using generative AI tooling?
   
   <!--
   If generative AI tooling has been used in the process of authoring this patch, please include the
   phrase: 'Generated-by: ' followed by the name of the tool and its version.
   If no, write 'No'.
   Please refer to the [ASF Generative Tooling Guidance](https://www.apache.org/legal/generative-tooling.html) for details.
   -->
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [VL] Add Velox batch resizer copyRanges fast path [gluten]

Posted by "yaooqinn (via GitHub)" <gi...@apache.org>.
yaooqinn commented on code in PR #12101:
URL: https://github.com/apache/gluten/pull/12101#discussion_r3265142986


##########
cpp/velox/benchmarks/VeloxBatchResizerBenchmark.cc:
##########
@@ -0,0 +1,568 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include <limits>
+#include <memory>
+#include <optional>
+#include <string>
+#include <vector>
+
+#include <arrow/util/bit_util.h>
+#include <arrow/util/bitmap_ops.h>
+#include <benchmark/benchmark.h>
+
+#include "memory/ColumnarBatchIterator.h"
+#include "memory/VeloxColumnarBatch.h"
+#include "shuffle/Payload.h"
+#include "utils/Exception.h"
+#include "utils/VeloxBatchResizer.h"
+#include "velox/common/memory/Memory.h"
+#include "velox/vector/ComplexVector.h"
+#include "velox/vector/FlatVector.h"
+
+using namespace facebook::velox;
+
+namespace gluten {
+namespace {
+
+constexpr int32_t kInputBatches = 64;
+constexpr int32_t kRowsPerBatch = 64;
+constexpr int32_t kTotalRows = kInputBatches * kRowsPerBatch;
+constexpr int64_t kPreferredBatchBytes = std::numeric_limits<int64_t>::max();

Review Comment:
   The 7 benchmark scenarios are all dense-flat, but `enableCopyRanges=true` (default) silently routes dictionary / constant / topLevelNull inputs through `RowVector::copy` (`VeloxBatchResizer.cc:148`). Suggest adding 1 dict-heavy + 1 constant-heavy scenario to validate the fallback path is at least perf-neutral before flipping the default on.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [VL] Add Velox batch resizer copyRanges fast path [gluten]

Posted by "zhli1142015 (via GitHub)" <gi...@apache.org>.
zhli1142015 commented on code in PR #12101:
URL: https://github.com/apache/gluten/pull/12101#discussion_r3265402305


##########
cpp/velox/benchmarks/VeloxBatchResizerBenchmark.cc:
##########
@@ -0,0 +1,568 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include <limits>
+#include <memory>
+#include <optional>
+#include <string>
+#include <vector>
+
+#include <arrow/util/bit_util.h>
+#include <arrow/util/bitmap_ops.h>
+#include <benchmark/benchmark.h>
+
+#include "memory/ColumnarBatchIterator.h"
+#include "memory/VeloxColumnarBatch.h"
+#include "shuffle/Payload.h"
+#include "utils/Exception.h"
+#include "utils/VeloxBatchResizer.h"
+#include "velox/common/memory/Memory.h"
+#include "velox/vector/ComplexVector.h"
+#include "velox/vector/FlatVector.h"
+
+using namespace facebook::velox;
+
+namespace gluten {
+namespace {
+
+constexpr int32_t kInputBatches = 64;
+constexpr int32_t kRowsPerBatch = 64;
+constexpr int32_t kTotalRows = kInputBatches * kRowsPerBatch;
+constexpr int64_t kPreferredBatchBytes = std::numeric_limits<int64_t>::max();

Review Comment:
   
      I added two fallback-focused benchmark scenarios for the copyRanges default-on path:
   
      - `DictionaryHeavy_64x64`
      - `ConstantHeavy_64x64`
   
      Each scenario compares:
   
      1. `BM_VeloxBatchResizerFallbackAppendOptOutBaseline`: `enableCopyRanges=false`, the old incremental append path.
      2. `BM_VeloxBatchResizerDefaultCopyRangesFallback`: default `enableCopyRanges=true`; dictionary/constant inputs are unsupported by child `copyRanges`, so   they fall back to `RowVector::copy` inside the new collect-and-copy path.
   
      Benchmark command:
   
      ```bash
      /tmp/gluten-cpp-ut-build/velox/benchmarks/velox_batch_resizer_benchmark \
        --benchmark_min_time=0.05s \
        --benchmark_repetitions=3 \
        --benchmark_format=json \
        > /tmp/velox_batch_resizer_perf.json
   
     Environment:
   
      - CPU: 32 logical CPUs @ ~2995 MHz
      - Caches: L1D 48 KiB x16, L1I 32 KiB x16, L2 2 MiB x16, L3 36 MiB x1
      - Note: ASLR was enabled, so the numbers may contain normal benchmark noise.
   
     Dense flat scenarios, CPU time averaged over 3 repetitions:
   
     ┌──────────────────┬─────────────────────────┬────────────────────┬─────────┬─────────────────────────┬─────────────────────────────┐
     │ Scenario         │ Append opt-out baseline │ Default copyRanges │ Speedup │ Direct child copyRanges │ Raw payload bulk-copy model │
     ├──────────────────┼─────────────────────────┼────────────────────┼─────────┼─────────────────────────┼─────────────────────────────┤
     │ Mixed_64x64      │ 94.3 us                 │ 11.3 us            │ 8.35x   │ 14.4 us                 │ 34.8 us                     │
     ├──────────────────┼─────────────────────────┼────────────────────┼─────────┼─────────────────────────┼─────────────────────────────┤
     │ Mixed_16x256     │ 31.7 us                 │ 6.3 us             │ 5.01x   │ 5.3 us                  │ 11.2 us                     │
     ├──────────────────┼─────────────────────────┼────────────────────┼─────────┼─────────────────────────┼─────────────────────────────┤
     │ Mixed_256x16     │ 234.2 us                │ 48.7 us            │ 4.81x   │ 30.6 us                 │ 119.2 us                    │
     ├──────────────────┼─────────────────────────┼────────────────────┼─────────┼─────────────────────────┼─────────────────────────────┤
     │ Fixed2_64x64     │ 29.3 us                 │ 5.4 us             │ 5.43x   │ 2.1 us                  │ 13.4 us                     │
     ├──────────────────┼─────────────────────────┼────────────────────┼─────────┼─────────────────────────┼─────────────────────────────┤
     │ Fixed16_64x64    │ 123.5 us                │ 26.7 us            │ 4.62x   │ 18.6 us                 │ 95.7 us                     │
     ├──────────────────┼─────────────────────────┼────────────────────┼─────────┼─────────────────────────┼─────────────────────────────┤
     │ LongString_64x64 │ 33.3 us                 │ 7.2 us             │ 4.61x   │ 4.1 us                  │ 15.7 us                     │
     ├──────────────────┼─────────────────────────┼────────────────────┼─────────┼─────────────────────────┼─────────────────────────────┤
     │ BoolHeavy_64x64  │ 73.4 us                 │ 10.8 us            │ 6.78x   │ 5.4 us                  │ 38.1 us                     │
     └──────────────────┴─────────────────────────┴────────────────────┴─────────┴─────────────────────────┴─────────────────────────────┘
   
     Fallback scenarios requested by the review comment, CPU time averaged over 3 repetitions:
   
     ┌───────────────────────┬─────────────────────────┬─────────────────────────────┬──────────────┐
     │ Scenario              │ Append opt-out baseline │ Default copyRanges fallback │ Result       │
     ├───────────────────────┼─────────────────────────┼─────────────────────────────┼──────────────┤
     │ DictionaryHeavy_64x64 │ 102.7 us                │ 77.7 us                     │ 1.32x faster │
     ├───────────────────────┼─────────────────────────┼─────────────────────────────┼──────────────┤
     │ ConstantHeavy_64x64   │ 67.8 us                 │ 42.0 us                     │ 1.61x faster │
     └───────────────────────┴─────────────────────────┴─────────────────────────────┴──────────────┘
   
     The fallback numbers are not regressions. The reason is that enableCopyRanges=true fallback is not the same as the old enableCopyRanges=false append path.
   
     With enableCopyRanges=false, VeloxBatchResizer creates an empty output RowVector and incrementally appends each small input batch via  buffer->append(input). That path grows and updates the output vector batch-by-batch.
   
     With enableCopyRanges=true, even when a batch is not eligible for child-vector copyRanges because it contains dictionary or constant children, the resizer   first collects the small input batches, computes the final totalRows, allocates/resizes the output RowVector once, and then copies each unsupported input   into the pre-sized output using:
   
      buffer->copy(input.get(), offset, 0, input->size());
   
     So these fallback benchmarks still benefit from the new collect-and-copy structure:
   
      - one output allocation / resize for the final row count;
      - fixed-offset RowVector::copy into a pre-sized output;
      - less incremental append bookkeeping;
      - Velox copy still has encoded-vector handling for dictionary and constant inputs, even though child copyRanges is not used.
   
     This explains why the default copyRanges-enabled fallback path is faster than the append opt-out baseline in these two scenarios. The important validation   point for the review is that dictionary-heavy and constant-heavy inputs do not show a perf regression after enabling copyRanges by default.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [VL] Add Velox batch resizer copyRanges fast path [gluten]

Posted by "yaooqinn (via GitHub)" <gi...@apache.org>.
yaooqinn merged PR #12101:
URL: https://github.com/apache/gluten/pull/12101


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org