You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2023/02/13 15:33:00 UTC
[spark] branch branch-3.4 updated: [SPARK-42420][CORE] Register WriteTaskResult, BasicWriteTaskStats, and ExecutedWriteSummary to KryoSerializer
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-3.4
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.4 by this push:
new 804c7f12e05 [SPARK-42420][CORE] Register WriteTaskResult, BasicWriteTaskStats, and ExecutedWriteSummary to KryoSerializer
804c7f12e05 is described below
commit 804c7f12e055071b56a7bc8ec027b7add8ed6a52
Author: William Hyun <wi...@apache.org>
AuthorDate: Mon Feb 13 07:32:27 2023 -0800
[SPARK-42420][CORE] Register WriteTaskResult, BasicWriteTaskStats, and ExecutedWriteSummary to KryoSerializer
### What changes were proposed in this pull request?
This PR aims to register `WriteTaskResult`, `BasicWriteTaskStats`, and `ExecutedWriteSummary` to `KryoSerializer`.
### Why are the changes needed?
This is used during the writing operation.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass the CIs.
Closes #39993 from williamhyun/kryowrite.
Authored-by: William Hyun <wi...@apache.org>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
(cherry picked from commit de0052ded40617f817bd71464c73ff7ee516b115)
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala | 3 +++
1 file changed, 3 insertions(+)
diff --git a/core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala b/core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala
index 307d55892aa..d79f6453bc5 100644
--- a/core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala
+++ b/core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala
@@ -543,6 +543,9 @@ private[serializer] object KryoSerializer {
"org.apache.spark.sql.types.Decimal$DecimalAsIfIntegral$",
"org.apache.spark.sql.types.Decimal$DecimalIsFractional$",
"org.apache.spark.sql.execution.command.PartitionStatistics",
+ "org.apache.spark.sql.execution.datasources.BasicWriteTaskStats",
+ "org.apache.spark.sql.execution.datasources.ExecutedWriteSummary",
+ "org.apache.spark.sql.execution.datasources.WriteTaskResult",
"org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTaskResult",
"org.apache.spark.sql.execution.joins.EmptyHashedRelation$",
"org.apache.spark.sql.execution.joins.LongHashedRelation",
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org