You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@celeborn.apache.org by "RexXiong (via GitHub)" <gi...@apache.org> on 2025/08/01 04:10:29 UTC
Re: [PR] [CELEBORN-2085] Use a fixed buffer for flush copying to reduce GC [celeborn]
RexXiong commented on code in PR #3394:
URL: https://github.com/apache/celeborn/pull/3394#discussion_r2246851625
##########
common/src/main/scala/org/apache/celeborn/common/CelebornConf.scala:
##########
@@ -6613,4 +6615,16 @@ object CelebornConf extends Logging {
.version("0.6.0")
.booleanConf
.createWithDefaultString("false")
+
+ val WORKER_FLUSH_REUSE_COPY_BUFFER_ENABLED: ConfigEntry[Boolean] =
+ buildConf("worker.flush.reuse.copy.buffer.enabled")
Review Comment:
`worker.flush.reuse.copy.buffer.enabled` -> `celeborn.worker.flush.reuseCopyBuffer.enabled`
##########
common/src/main/scala/org/apache/celeborn/common/CelebornConf.scala:
##########
@@ -6613,4 +6615,16 @@ object CelebornConf extends Logging {
.version("0.6.0")
.booleanConf
.createWithDefaultString("false")
+
+ val WORKER_FLUSH_REUSE_COPY_BUFFER_ENABLED: ConfigEntry[Boolean] =
+ buildConf("worker.flush.reuse.copy.buffer.enabled")
+ .categories("worker")
+ .doc("Whether to enable reuse copy buffer for flush. Note that this copy buffer must not" +
+ " be referenced again after flushing. This means that, for example, the Hdfs(Oss or S3) client" +
+ " will not asynchronously access this buffer after the flush method returns, otherwise data" +
+ " modification problems will occur.")
+ .version("0.6.0")
Review Comment:
0.6.1?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org