You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2023/09/26 05:57:05 UTC
[spark] branch master updated: [SPARK-45333][CORE] Fix one unit mistake related to spark.eventLog.buffer.kb
This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 91b76070b05f [SPARK-45333][CORE] Fix one unit mistake related to spark.eventLog.buffer.kb
91b76070b05f is described below
commit 91b76070b05fdf026a41f22c12e404ee99bd8cc3
Author: lanmengran1 <la...@jd.com>
AuthorDate: Tue Sep 26 14:56:51 2023 +0900
[SPARK-45333][CORE] Fix one unit mistake related to spark.eventLog.buffer.kb
### What changes were proposed in this pull request?
Fixing a unit mistake in the usage of configuration "spark.eventLog.buffer.kb"
### Why are the changes needed?
Making the size of the event log output buffer as expected
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Maybe don't need
Closes #42294 from amoylan2/fix_unit_mistake_in_eventLogOutputBufferSize.
Authored-by: lanmengran1 <la...@jd.com>
Signed-off-by: Hyukjin Kwon <gu...@apache.org>
---
.../scala/org/apache/spark/deploy/history/EventLogFileWriters.scala | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/core/src/main/scala/org/apache/spark/deploy/history/EventLogFileWriters.scala b/core/src/main/scala/org/apache/spark/deploy/history/EventLogFileWriters.scala
index 144dadf29bc3..e7eb05c85367 100644
--- a/core/src/main/scala/org/apache/spark/deploy/history/EventLogFileWriters.scala
+++ b/core/src/main/scala/org/apache/spark/deploy/history/EventLogFileWriters.scala
@@ -57,7 +57,7 @@ abstract class EventLogFileWriter(
protected val shouldCompress = sparkConf.get(EVENT_LOG_COMPRESS) &&
!sparkConf.get(EVENT_LOG_COMPRESSION_CODEC).equalsIgnoreCase("none")
protected val shouldOverwrite = sparkConf.get(EVENT_LOG_OVERWRITE)
- protected val outputBufferSize = sparkConf.get(EVENT_LOG_OUTPUT_BUFFER_SIZE).toInt
+ protected val outputBufferSize = sparkConf.get(EVENT_LOG_OUTPUT_BUFFER_SIZE).toInt * 1024
protected val fileSystem = Utils.getHadoopFileSystem(logBaseDir, hadoopConf)
protected val compressionCodec =
if (shouldCompress) {
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org