You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by wa...@apache.org on 2025/06/09 05:28:34 UTC
(incubator-gluten) branch main updated: [GLUTEN-6269][VL][UNIFFLE] Remove inc BytesWritten in VeloxUniffleColumnarShuffleWriter (#9897)
This is an automated email from the ASF dual-hosted git repository.
wangzhen pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git
The following commit(s) were added to refs/heads/main by this push:
new 5c3f577eb1 [GLUTEN-6269][VL][UNIFFLE] Remove inc BytesWritten in VeloxUniffleColumnarShuffleWriter (#9897)
5c3f577eb1 is described below
commit 5c3f577eb13398509c90b2edd6a7774651d69faf
Author: Zhen Wang <64...@qq.com>
AuthorDate: Mon Jun 9 13:28:31 2025 +0800
[GLUTEN-6269][VL][UNIFFLE] Remove inc BytesWritten in VeloxUniffleColumnarShuffleWriter (#9897)
---
.../apache/spark/shuffle/writer/VeloxUniffleColumnarShuffleWriter.java | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/backends-velox/src-uniffle/main/java/org/apache/spark/shuffle/writer/VeloxUniffleColumnarShuffleWriter.java b/backends-velox/src-uniffle/main/java/org/apache/spark/shuffle/writer/VeloxUniffleColumnarShuffleWriter.java
index 6bdb5b54af..383c30939f 100644
--- a/backends-velox/src-uniffle/main/java/org/apache/spark/shuffle/writer/VeloxUniffleColumnarShuffleWriter.java
+++ b/backends-velox/src-uniffle/main/java/org/apache/spark/shuffle/writer/VeloxUniffleColumnarShuffleWriter.java
@@ -226,7 +226,8 @@ public class VeloxUniffleColumnarShuffleWriter<K, V> extends RssShuffleWriter<K,
- splitResult.getTotalWriteTime()
- splitResult.getTotalCompressTime());
- shuffleWriteMetrics.incBytesWritten(splitResult.getTotalBytesWritten());
+ // bytesWritten is calculated in uniffle side: WriteBufferManager.createShuffleBlock
+ // shuffleWriteMetrics.incBytesWritten(splitResult.getTotalBytesWritten());
shuffleWriteMetrics.incWriteTime(
splitResult.getTotalWriteTime() + splitResult.getTotalPushTime());
// partitionLengths is calculate in uniffle side
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org