[jira] [Created] (SPARK-42252) Deprecate spark.shuffle.unsafe.file.output.buffer and add a new config

["Wei Guo (Jira)" <ji...@apache.org>]

Wei Guo created SPARK-42252:
-------------------------------

             Summary: Deprecate spark.shuffle.unsafe.file.output.buffer and add a new config
                 Key: SPARK-42252
                 URL: https://issues.apache.org/jira/browse/SPARK-42252
             Project: Spark
          Issue Type: Improvement
          Components: Spark Core
    Affects Versions: 3.0.0
            Reporter: Wei Guo
             Fix For: 3.4.0


After Jira SPARK-28209 and PR [25007|[https://github.com/apache/spark/pull/25007]], the new shuffle writer api is proposed. All shuffle writers(BypassMergeSortShuffleWriter, SortShuffleWriter, UnsafeShuffleWriter) are based on LocalDiskShuffleMapOutputWriter to write local disk shuffle files. The config spark.shuffle.unsafe.file.output.buffer used in LocalDiskShuffleMapOutputWriter was only used in UnsafeShuffleWriter before. 
 
It's better to rename it and make it more suitable.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org