You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by yu...@apache.org on 2026/08/13 19:59:06 UTC

(gluten) branch main updated: [GLUTEN-6887][VL] Daily Update Velox Version (2026_08_12) (#12755)

This is an automated email from the ASF dual-hosted git repository.

zhouyuan pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/gluten.git


The following commit(s) were added to refs/heads/main by this push:
     new cd22fb88f5 [GLUTEN-6887][VL] Daily Update Velox Version (2026_08_12) (#12755)
cd22fb88f5 is described below

commit cd22fb88f521133570f3b20853ce77566788b171
Author: Gluten Performance Bot <13...@users.noreply.github.com>
AuthorDate: Thu Aug 13 20:59:00 2026 +0100

    [GLUTEN-6887][VL] Daily Update Velox Version (2026_08_12) (#12755)
    
    * [GLUTEN-6887][VL] Daily Update Velox Version (dft-2026_08_12)
    
    Upstream Velox's New Commits:
    55acc87ce by Ismaël Mejía, perf(parquet): Dictionary passthrough and selective flattening in Parquet writer
    8f2e1ae75 by PRASHANT GOLASH, feat(nimble): Add reader-owned lazy I/O load hook (#17999)
    844c77cc9 by oerling, feat(torchwave): Multikernel parity with cg (#18390)
    f0d0557ed by Pedro Pedreira, fix(io_uring): Handle ENOMEM from the availability probe
    b3701be01 by wankun, fix: Prevent cancelled direct coalesced loads from exposing partial data
    1d8d1e291 by LingBin, refactor: Use CHECK_AND_INC_MEM_OP_STATS for external alloc/free stats
    d3d59c862 by carlnayak, feat: Add xxhash128 Presto scalar function (#18436)
    fc99399aa by Guilherme Kunigami, feat(remote): Only send selected rows to the remote server (#18379)
    91b49e730 by Krishnakanth, feat: Support kAnti and kLeftSemiFilter in NestedLoopJoin (#18161)
    5017bfaf1 by Pedro Pedreira, fix(vector): Support OPAQUE vectors in saveVector/restoreVector
    0814ab6c8 by dependabot[bot], build(deps): Bump fast-uri from 3.1.0 to 3.1.5 in /website
    1d6881b4e by RindsSchei225e, refactor(writer): Rename VeloxWriter to Writer (#18431)
    b1261a0e9 by Shrinidhi Joshi, feat(exchange): Add TransportKind::kMaterialized and TransportOptions to PartitionedOutput (#18425)
    36a0651a7 by Krishna Pai, fix(ci): Bump Ubuntu tzdata pin to the currently published build (fixes #18440)
    deca9543f by dependabot[bot], build(deps): Bump websocket-driver from 0.7.4 to 0.7.5 in /website
    a0bf9327b by Andrii Rosa, fix: Always initialize shell for uv tool
    351b0a72f by Sreeni Viswanadha, perf: Skip string conversion for single-element array_join (#18361)
    71cd8338c by Matt Gara, fix(cudf): Synchronize MarkDistinct state across streams
    b603b5b26 by Muhammad Haseeb, feat(cudf): Transform and push down filters in cuDF Iceberg
    27196fead by Matt Gara, fix(cudf): Preserve zero-column nested loop join row counts
    d4b5285d5 by Xiaoxuan, feat: Add shared dictionary builder and encoding (#18418)
    fd6fbc5e9 by Pedro Pedreira, fix(fuzzer): Cap KHyperLogLog generated sketch size
    a6b2a8f29 by Bikramjeet Singh Vig, fix(dwrf): Apply key filters to preserved flat maps
    92422486e by Minni Mittal, feat(spark): Add randn scalar function
    77558daec by Raymond Lin, feat(hive): Add read bytes and format to scan batch event (#18407)
    6d0667cb7 by dependabot[bot], build(deps): Bump postcss from 8.5.12 to 8.5.25 in /website
    568652c5f by Kunal Pathak, perf(vector): Accumulate consecutive copy ranges in locals in copyRangesImpl
    5cbf734d1 by LingBin, refactor(memory): Remove dangling comment in SharedArbitrator.h
    9f4cf9938 by Hongze Zhang, perf: Optimize probing misses using bloom filter for kLeft / kLeftSemiProject / kAnti hash join
    d3c2c8cd6 by LingBin, docs(memory): Fix and clarify ArbitrationOperation timestamp comments
    a34b683f4 by Natasha Sehgal, feat(aggregates): VARBINARY compare in N-arg min_by/max_by
    1354a214b by Natasha Sehgal, test(fuzzer): Fuzz qdigest scalars over full value range (#18412)
    ef92cf1ba by zhichenxu-meta, fix(rpc): Park instead of busy-spinning under mid-stream BATCH back-pressure (#18300)
    2615dde9d by JiaKe, refactor(hashjoin): Refactor hashJoinTableType and isLeftNullAwareJoinWithFilter API in HashJoinBridge
    c1652ed62 by oerling, feat(torchwave): Fused in-place slice/select/scatter ops (#18389)
    908bf838e by abhinavmuk04, feat: Add Layer 2 AIMD-under-load test for RPC framework (#18311)
    
    Signed-off-by: glutenperfbot <gl...@glutenproject-internal.com>
    
    * exclude UT for now
    
    Signed-off-by: Yuan <yu...@apache.org>
    
    * fix
    
    Signed-off-by: Yuan <yu...@apache.org>
    
    ---------
    
    Signed-off-by: glutenperfbot <gl...@glutenproject-internal.com>
    Signed-off-by: Yuan <yu...@apache.org>
    Co-authored-by: glutenperfbot <gl...@glutenproject-internal.com>
    Co-authored-by: Yuan <yu...@apache.org>
---
 ep/build-velox/src/get-velox.sh                                   | 4 ++--
 .../scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala   | 4 ++++
 .../scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala   | 8 ++++++++
 .../scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala   | 8 ++++++++
 4 files changed, 22 insertions(+), 2 deletions(-)

diff --git a/ep/build-velox/src/get-velox.sh b/ep/build-velox/src/get-velox.sh
index 5df0b0c28a..477b75cf30 100755
--- a/ep/build-velox/src/get-velox.sh
+++ b/ep/build-velox/src/get-velox.sh
@@ -18,8 +18,8 @@ set -exu
 
 CURRENT_DIR=$(cd "$(dirname "$BASH_SOURCE")"; pwd)
 VELOX_REPO=https://github.com/IBM/velox.git
-VELOX_BRANCH=dft-2026_08_05
-VELOX_ENHANCED_BRANCH=ibm-2026_08_05
+VELOX_BRANCH=dft-2026_08_12
+VELOX_ENHANCED_BRANCH=ibm-2026_08_12
 VELOX_HOME=""
 RUN_SETUP_SCRIPT=ON
 ENABLE_ENHANCED_FEATURES=OFF
diff --git a/gluten-ut/spark35/src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala b/gluten-ut/spark35/src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala
index e7a8c93ba8..22c72ba392 100644
--- a/gluten-ut/spark35/src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala
+++ b/gluten-ut/spark35/src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala
@@ -628,6 +628,8 @@ class VeloxTestSettings extends BackendTestSettings {
     .exclude("length check for input string values: nested in map value")
     .exclude("length check for input string values: nested in both map key and value")
     .exclude("length check for input string values: nested in array of struct")
+    .excludeGlutenTest("length check for input string values: nested in array of struct")
+    .exclude("char type values should be padded: nested in array of struct")
     .exclude("length check for input string values: nested in array of array")
     // Following tests are excluded as these are overridden in Gluten test suite..
     // The overridden tests assert against Velox-specific error messages for char/varchar
@@ -645,6 +647,8 @@ class VeloxTestSettings extends BackendTestSettings {
     .exclude("length check for input string values: nested in array")
     .exclude("length check for input string values: nested in struct of array")
     .exclude("length check for input string values: nested in array of struct")
+    .excludeGlutenTest("length check for input string values: nested in array of struct")
+    .exclude("char type values should be padded: nested in array of struct")
     .exclude("length check for input string values: nested in array of array")
     .exclude("length check for input string values: with implicit cast")
     .exclude("char/varchar type values length check: partitioned columns of other types")
diff --git a/gluten-ut/spark40/src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala b/gluten-ut/spark40/src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala
index f12483655b..2e068639f9 100644
--- a/gluten-ut/spark40/src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala
+++ b/gluten-ut/spark40/src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala
@@ -949,7 +949,15 @@ class VeloxTestSettings extends BackendTestSettings {
     // Rewrite for different cache size.
     .exclude("SPARK-36120: Support cache/uncache table with TimestampNTZ type")
   enableSuite[GlutenFileSourceCharVarcharTestSuite]
+    // Velox Parquet writer (post dft-2026_08_12) introduced selective per-column flattening
+    // that misses CONSTANT-encoded descendants inside nested complex types (ARRAY<STRUCT>).
+    // Writes to ARRAY<STRUCT<char/varchar>> crash with VeloxRuntimeError INVALID_STATE in the
+    // Arrow bridge instead of producing the expected error or result.
+    .exclude("length check for input string values: nested in array of struct")
+    .exclude("char type values should be padded: nested in array of struct")
   enableSuite[GlutenDSV2CharVarcharTestSuite]
+    .exclude("length check for input string values: nested in array of struct")
+    .exclude("char type values should be padded: nested in array of struct")
   enableSuite[GlutenColumnExpressionSuite]
     // Velox raise_error('errMsg') throws a velox_user_error exception with the message 'errMsg'.
     // The final caught Spark exception's getCause().getMessage() contains 'errMsg' but does not
diff --git a/gluten-ut/spark41/src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala b/gluten-ut/spark41/src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala
index b196f7351f..e027321f18 100644
--- a/gluten-ut/spark41/src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala
+++ b/gluten-ut/spark41/src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala
@@ -933,7 +933,15 @@ class VeloxTestSettings extends BackendTestSettings {
     .exclude("SPARK-36120: Support cache/uncache table with TimestampNTZ type")
   enableSuite[GlutenCacheTableInKryoSuite]
   enableSuite[GlutenFileSourceCharVarcharTestSuite]
+    // Velox Parquet writer (post dft-2026_08_12) introduced selective per-column flattening
+    // that misses CONSTANT-encoded descendants inside nested complex types (ARRAY<STRUCT>).
+    // Writes to ARRAY<STRUCT<char/varchar>> crash with VeloxRuntimeError INVALID_STATE in the
+    // Arrow bridge instead of producing the expected error or result.
+    .exclude("length check for input string values: nested in array of struct")
+    .exclude("char type values should be padded: nested in array of struct")
   enableSuite[GlutenDSV2CharVarcharTestSuite]
+    .exclude("length check for input string values: nested in array of struct")
+    .exclude("char type values should be padded: nested in array of struct")
   enableSuite[GlutenColumnExpressionSuite]
     // Velox raise_error('errMsg') throws a velox_user_error exception with the message 'errMsg'.
     // The final caught Spark exception's getCause().getMessage() contains 'errMsg' but does not


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org