You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by yu...@apache.org on 2026/10/13 18:39:26 UTC
(gluten) branch main updated: [GLUTEN-6887][VL] Daily Update Velox Version (2026_10_13) (#13257)
This is an automated email from the ASF dual-hosted git repository.
zhouyuan pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/gluten.git
The following commit(s) were added to refs/heads/main by this push:
new 4c1a850639 [GLUTEN-6887][VL] Daily Update Velox Version (2026_10_13) (#13257)
4c1a850639 is described below
commit 4c1a8506396a0626707765413b7f2c58d273cbdb
Author: Gluten Performance Bot <13...@users.noreply.github.com>
AuthorDate: Tue Oct 13 19:39:19 2026 +0100
[GLUTEN-6887][VL] Daily Update Velox Version (2026_10_13) (#13257)
* [GLUTEN-6887][VL] Daily Update Velox Version (dft-2026_10_13)
Upstream Velox's New Commits:
b918c63dc by Apurva Kumar, fix(iceberg): Use split values for identity partition columns
0382c5b76 by Xiaoxuan, feat(nimble): Add index-backed row and batch iterators (#19520)
34c0db3d0 by Karthikeyan, refactor(functions): Move date/time unit arithmetic to a GPU header
aeeaf8816 by macduan, feat(nimble): Add ALPRD bulk reads and encoding views
700554055 by Ke, feat(nimble): Reuse dictionary visitor for numeric reads (#19254)
081ac3c26 by Pedro Pedreira, refactor(vector): Use fromMapVector and assertEqualVectors in tests
c22e08eb3 by Minni Mittal, feat(spark): Add initial format_string and printf support
4eb15050d by Karthikeyan, fix(cudf): Label CudfFromVelox output with its input type
0adb32470 by Maria Basmanova, fix(type): Delete inherited create() in derived scalar types (#19526)
9f290056e by Minni Mittal, feat(spark): Add octet_length scalar function
5271a1e6d by Mohammad Linjawi, refactor(spark): Move hashOne overloads to Hash.h
aa5020aef by Shruti Shivakumar, feat(cudf): Enable streaming final decimal SUM and AVG
315b76984 by Karthikeyan, feat(cudf): Pass the session QueryConfig to evaluator creation
b19d44974 by Patrick Wilson, fix(cudf): Wait for probe reads before releasing hash join state
4831b2dba by PRASHANT GOLASH, feat(nimble): Add FSST EncodingView support (#19397)
c0225a65f by XiaoDu, feat(nimble): Add IVF RaBitQ FastScan (#19405)
18fa49af5 by PRASHANT GOLASH, refactor(nimble): Factor out driver context setup for parallel encoding (#19410)
f92d61d1c by Sahil Gandhi, feat(exec): Report experimental fixed-point execution stats (#19455)
0b3944f98 by Natasha Sehgal, test(functions): Re-enable Poisson CDF parity fuzzing
1625bdb1a by Sahil Gandhi, perf(exec): Build fixed-point hash tables per driver (#19454)
e216da572 by XiaoDu, fix(nimble): Honor vector index pinning policy (#19515)
c37a5ce37 by Sahil Gandhi, fix(exec): Reject grouped execution for fixed points (#19453)
780d58a0c by XiaoDu, feat(nimble): Parallelize IVF query-list scans (#19399)
f5c38f87b by XiaoDu, feat(nimble): Push row filters into vector search (#19321)
5402a01c0 by Pedro Pedreira, feat(vector): Add FlatMapVector::fromMapVector() (#19507)
1d2534421 by Xiaoxuan, refactor(nimble): Remove `allowNestedAlpSelection`
52827270a by Suryadev Sahadevan Rajesh, feat(nimble): Auto-select SubIntSplit child encodings and split boundaries (#19504)
71e04d9e9 by tanjialiang, feat(nimble): Store per stream checksums in stream trailers
6c9c8bad7 by Pedro Pedreira, fix(vector): Fix flat map copies in encodedVectorCopy (#19499)
6cbe2d062 by Pedro Pedreira, fix(functions): Respect in-map bits in element_at on flat maps (#19500)
010931b18 by Pedro Pedreira, feat(vector): Copy between MapVector and FlatMapVector (#19486)
Signed-off-by: glutenperfbot <gl...@glutenproject-internal.com>
* disable spark-40 tests in enhanced branch to save some credit
Signed-off-by: Yuan <yu...@apache.org>
---------
Signed-off-by: glutenperfbot <gl...@glutenproject-internal.com>
Signed-off-by: Yuan <yu...@apache.org>
Co-authored-by: glutenperfbot <gl...@glutenproject-internal.com>
Co-authored-by: Yuan <yu...@apache.org>
---
.github/workflows/velox_backend_enhanced.yml | 130 +++++++++++++--------------
ep/build-velox/src/get-velox.sh | 4 +-
2 files changed, 67 insertions(+), 67 deletions(-)
diff --git a/.github/workflows/velox_backend_enhanced.yml b/.github/workflows/velox_backend_enhanced.yml
index e84de81c4e..a38e890066 100644
--- a/.github/workflows/velox_backend_enhanced.yml
+++ b/.github/workflows/velox_backend_enhanced.yml
@@ -229,68 +229,68 @@ jobs:
**/gluten-ut/**/hs_err_*.log
**/gluten-ut/**/core.*
- spark-test-spark40:
- needs: [detect-changes, build-native-lib-centos-8]
- if: >-
- needs.detect-changes.outputs.java == 'true' ||
- needs.detect-changes.outputs.shims40 == 'true' ||
- needs.detect-changes.outputs.cpp == 'true'
- runs-on: ubuntu-22.04
- strategy:
- fail-fast: false
- matrix:
- group: [1, 2, 3]
- env:
- SPARK_TESTING: true
- container: apache/gluten:centos-8-jdk17
- steps:
- - uses: actions/checkout@v7
- - name: Download All Artifacts
- uses: actions/download-artifact@v8
- with:
- name: velox-native-lib-enhanced-centos-8-${{github.sha}}
- path: ./cpp/build/releases/
- - name: Prepare
- run: |
- dnf module -y install python39 && \
- alternatives --set python3 /usr/bin/python3.9 && \
- pip3 install setuptools==77.0.3 && \
- pip3 install pyspark==3.5.5 cython && \
- pip3 install pandas==2.2.3 pyarrow==20.0.0
- - name: Build and Run unit test for Spark 4.0.0 with scala-2.13 (other tests, group ${{ matrix.group }})
- run: |
- cd $GITHUB_WORKSPACE/
- export SPARK_SCALA_VERSION=2.13
- yum install -y java-17-openjdk-devel
- export JAVA_HOME=/usr/lib/jvm/java-17-openjdk
- export PATH=$JAVA_HOME/bin:$PATH
- java -version
- TAGS_EXCLUDE="org.apache.spark.tags.ExtendedSQLTest,org.apache.spark.tags.SlowHiveTest,org.apache.gluten.tags.UDFTest,org.apache.gluten.tags.CudfTest,org.apache.gluten.tags.SkipTest"
- if [ "${{ matrix.group }}" = "1" ]; then
- $MVN_CMD clean test -Pspark-4.0 -Pscala-2.13 -Pjava-17 -Pbackends-velox -Piceberg -Pdelta \
- -Pspark-ut -DargLine="-Dspark.test.home=/opt/shims/spark40/spark_home/" -DtagsToExclude="$TAGS_EXCLUDE" \
- -DwildcardSuites="org.apache.spark.sql.streaming,org.apache.spark.GlutenSortShuffleSuite,org.apache.gluten"
- elif [ "${{ matrix.group }}" = "2" ]; then
- $MVN_CMD clean test -Pspark-4.0 -Pscala-2.13 -Pjava-17 -Pbackends-velox -Piceberg -Pdelta \
- -Pspark-ut -DargLine="-Dspark.test.home=/opt/shims/spark40/spark_home/" -DtagsToExclude="$TAGS_EXCLUDE" \
- -DwildcardSuites="org.apache.spark.sql.execution,org.apache.spark.sql.catalyst,org.apache.spark.sql.errors,org.apache.spark.sql.extension"
- else
- $MVN_CMD clean test -Pspark-4.0 -Pscala-2.13 -Pjava-17 -Pbackends-velox -Piceberg -Pdelta \
- -Pspark-ut -DargLine="-Dspark.test.home=/opt/shims/spark40/spark_home/" -DtagsToExclude="$TAGS_EXCLUDE" \
- -DwildcardSuites="org.apache.spark.sql.GlutenSQL,org.apache.spark.sql.Gluten,org.apache.spark.sql.connector,org.apache.spark.sql.sources,org.apache.spark.sql.hive,org.apache.spark.sql.gluten,org.apache.spark.sql.shim"
- fi
- - name: Upload test report
- if: always()
- uses: actions/upload-artifact@v7
- with:
- name: spark-test-spark40-group${{ matrix.group }}-report
- path: '**/surefire-reports/TEST-*.xml'
- - name: Upload unit tests log files
- if: ${{ !success() }}
- uses: actions/upload-artifact@v7
- with:
- name: spark-test-spark40-group${{ matrix.group }}-test-log
- path: |
- **/target/*.log
- **/gluten-ut/**/hs_err_*.log
- **/gluten-ut/**/core.*
+ # spark-test-spark40:
+ # needs: [detect-changes, build-native-lib-centos-8]
+ # if: >-
+ # needs.detect-changes.outputs.java == 'true' ||
+ # needs.detect-changes.outputs.shims40 == 'true' ||
+ # needs.detect-changes.outputs.cpp == 'true'
+ # runs-on: ubuntu-22.04
+ # strategy:
+ # fail-fast: false
+ # matrix:
+ # group: [1, 2, 3]
+ # env:
+ # SPARK_TESTING: true
+ # container: apache/gluten:centos-8-jdk17
+ # steps:
+ # - uses: actions/checkout@v7
+ # - name: Download All Artifacts
+ # uses: actions/download-artifact@v8
+ # with:
+ # name: velox-native-lib-enhanced-centos-8-${{github.sha}}
+ # path: ./cpp/build/releases/
+ # - name: Prepare
+ # run: |
+ # dnf module -y install python39 && \
+ # alternatives --set python3 /usr/bin/python3.9 && \
+ # pip3 install setuptools==77.0.3 && \
+ # pip3 install pyspark==3.5.5 cython && \
+ # pip3 install pandas==2.2.3 pyarrow==20.0.0
+ # - name: Build and Run unit test for Spark 4.0.0 with scala-2.13 (other tests, group ${{ matrix.group }})
+ # run: |
+ # cd $GITHUB_WORKSPACE/
+ # export SPARK_SCALA_VERSION=2.13
+ # yum install -y java-17-openjdk-devel
+ # export JAVA_HOME=/usr/lib/jvm/java-17-openjdk
+ # export PATH=$JAVA_HOME/bin:$PATH
+ # java -version
+ # TAGS_EXCLUDE="org.apache.spark.tags.ExtendedSQLTest,org.apache.spark.tags.SlowHiveTest,org.apache.gluten.tags.UDFTest,org.apache.gluten.tags.CudfTest,org.apache.gluten.tags.SkipTest"
+ # if [ "${{ matrix.group }}" = "1" ]; then
+ # $MVN_CMD clean test -Pspark-4.0 -Pscala-2.13 -Pjava-17 -Pbackends-velox -Piceberg -Pdelta \
+ # -Pspark-ut -DargLine="-Dspark.test.home=/opt/shims/spark40/spark_home/" -DtagsToExclude="$TAGS_EXCLUDE" \
+ # -DwildcardSuites="org.apache.spark.sql.streaming,org.apache.spark.GlutenSortShuffleSuite,org.apache.gluten"
+ # elif [ "${{ matrix.group }}" = "2" ]; then
+ # $MVN_CMD clean test -Pspark-4.0 -Pscala-2.13 -Pjava-17 -Pbackends-velox -Piceberg -Pdelta \
+ # -Pspark-ut -DargLine="-Dspark.test.home=/opt/shims/spark40/spark_home/" -DtagsToExclude="$TAGS_EXCLUDE" \
+ # -DwildcardSuites="org.apache.spark.sql.execution,org.apache.spark.sql.catalyst,org.apache.spark.sql.errors,org.apache.spark.sql.extension"
+ # else
+ # $MVN_CMD clean test -Pspark-4.0 -Pscala-2.13 -Pjava-17 -Pbackends-velox -Piceberg -Pdelta \
+ # -Pspark-ut -DargLine="-Dspark.test.home=/opt/shims/spark40/spark_home/" -DtagsToExclude="$TAGS_EXCLUDE" \
+ # -DwildcardSuites="org.apache.spark.sql.GlutenSQL,org.apache.spark.sql.Gluten,org.apache.spark.sql.connector,org.apache.spark.sql.sources,org.apache.spark.sql.hive,org.apache.spark.sql.gluten,org.apache.spark.sql.shim"
+ # fi
+ # - name: Upload test report
+ # if: always()
+ # uses: actions/upload-artifact@v7
+ # with:
+ # name: spark-test-spark40-group${{ matrix.group }}-report
+ # path: '**/surefire-reports/TEST-*.xml'
+ # - name: Upload unit tests log files
+ # if: ${{ !success() }}
+ # uses: actions/upload-artifact@v7
+ # with:
+ # name: spark-test-spark40-group${{ matrix.group }}-test-log
+ # path: |
+ # **/target/*.log
+ # **/gluten-ut/**/hs_err_*.log
+ # **/gluten-ut/**/core.*
diff --git a/ep/build-velox/src/get-velox.sh b/ep/build-velox/src/get-velox.sh
index 71f1610c3b..de4b1063eb 100755
--- a/ep/build-velox/src/get-velox.sh
+++ b/ep/build-velox/src/get-velox.sh
@@ -18,8 +18,8 @@ set -exu
CURRENT_DIR=$(cd "$(dirname "$BASH_SOURCE")"; pwd)
VELOX_REPO=https://github.com/IBM/velox.git
-VELOX_BRANCH=dft-2026_10_10
-VELOX_ENHANCED_BRANCH=ibm-2026_10_10
+VELOX_BRANCH=dft-2026_10_13
+VELOX_ENHANCED_BRANCH=ibm-2026_10_13
VELOX_HOME=""
RUN_SETUP_SCRIPT=ON
ENABLE_ENHANCED_FEATURES=OFF
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org