[jira] [Updated] (SPARK-56496) maxConsecutiveStageAttempts vs maxStageAttempts dual-limit inconsistency

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-56496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated SPARK-56496:
-----------------------------------
    Labels: pull-request-available  (was: )

> maxConsecutiveStageAttempts vs maxStageAttempts dual-limit inconsistency
> ------------------------------------------------------------------------
>
>                 Key: SPARK-56496
>                 URL: https://issues.apache.org/jira/browse/SPARK-56496
>             Project: Spark
>          Issue Type: Sub-task
>          Components: Spark Core
>    Affects Versions: 4.1.1
>            Reporter: Xingbo Jiang
>            Assignee: Xingbo Jiang
>            Priority: Major
>              Labels: pull-request-available
>
> *Problem*
> DAGScheduler has two separate stage-retry limits: maxConsecutiveStageAttempts (for consecutive failures) and maxStageAttempts (total attempts). Different code paths check only one of the two limits, so the effective abort behavior depends on which path is taken. This can cause unexpected job aborts (stricter limit wins silently) or prevent expected aborts.
> *Impact*
> Unpredictable job abort behavior. Users configuring one limit may be surprised by the other. Inconsistent behavior between fetch-failure paths and non-fetch-failure paths.
> *Suggested Fix*
> Document the exact interaction (both limits are enforced independently; whichever triggers first aborts the stage). Unify the limit check into a single canRetryStage helper used by all paths.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org