[jira] [Created] (SPARK-56496) maxConsecutiveStageAttempts vs maxStageAttempts dual-limit inconsistency

["Xingbo Jiang (Jira)" <ji...@apache.org>]

Xingbo Jiang created SPARK-56496:
------------------------------------

             Summary: maxConsecutiveStageAttempts vs maxStageAttempts dual-limit inconsistency
                 Key: SPARK-56496
                 URL: https://issues.apache.org/jira/browse/SPARK-56496
             Project: Spark
          Issue Type: Sub-task
          Components: Spark Core
    Affects Versions: 4.1.1
            Reporter: Xingbo Jiang
            Assignee: Xingbo Jiang


*Problem*
DAGScheduler has two separate stage-retry limits: maxConsecutiveStageAttempts (for consecutive failures) and maxStageAttempts (total attempts). Different code paths check only one of the two limits, so the effective abort behavior depends on which path is taken. This can cause unexpected job aborts (stricter limit wins silently) or prevent expected aborts.

*Impact*
Unpredictable job abort behavior. Users configuring one limit may be surprised by the other. Inconsistent behavior between fetch-failure paths and non-fetch-failure paths.

*Suggested Fix*
Document the exact interaction (both limits are enforced independently; whichever triggers first aborts the stage). Unify the limit check into a single canRetryStage helper used by all paths.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org