[jira] [Created] (SPARK-49001) Add a config to add ability to only enable speculative execution on non-data-writing tasks.

["James Xu (Jira)" <ji...@apache.org>]

James Xu created SPARK-49001:
--------------------------------

             Summary: Add a config to add ability to only enable speculative execution on non-data-writing tasks.
                 Key: SPARK-49001
                 URL: https://issues.apache.org/jira/browse/SPARK-49001
             Project: Spark
          Issue Type: Improvement
          Components: Spark Core
    Affects Versions: 3.3.2
            Reporter: James Xu


Speculative Execution is a very good tool to avoid straggler tasks, but if we use Speculative Execution with some customized BatchWrite, if there are bugs in user implemented BatchWrite, it will produce wrong results.

 

So this ticket propose to add a config to add the ability to only enable speculative execution on non-data-writing tasks, which might be more safe.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org