You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Jungtaek Lim (Jira)" <ji...@apache.org> on 2026/05/15 19:05:00 UTC

[jira] [Resolved] (SPARK-56720) Fail subsequent async log writes after a prior failure in async progress tracking

     [ https://issues.apache.org/jira/browse/SPARK-56720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jungtaek Lim resolved SPARK-56720.
----------------------------------
    Fix Version/s: 4.3.0
                   5.0.0
       Resolution: Fixed

Issue resolved by pull request 55676
[https://github.com/apache/spark/pull/55676]

> Fail subsequent async log writes after a prior failure in async progress tracking
> ---------------------------------------------------------------------------------
>
>                 Key: SPARK-56720
>                 URL: https://issues.apache.org/jira/browse/SPARK-56720
>             Project: Spark
>          Issue Type: Bug
>          Components: Structured Streaming
>    Affects Versions: 3.4.0
>            Reporter: Yuchen Liu
>            Assignee: Yuchen Liu
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 4.3.0, 5.0.0
>
>
> When async progress tracking is enabled, offset and commit log writes are submitted to a single-threaded executor in {{AsyncOffsetSeqLog}} / {{{}AsyncCommitLog{}}}. If one async write task fails (e.g. an HDFS {{Permission denied}} or other {{{}IOException{}}}), follow-up tasks already queued — or queued before the main thread re-checks {{errorNotifier}} at the next batch boundary — still execute and may successfully persist files to durable storage. This produces two correctness/observability problems:
>  # Gaps on durable storage. The offset log may be missing batch _N_ while batch _N+1_ is present, or a commit-log entry can be written without its corresponding offset-log entry. This violates the invariant that the commit log is a prefix of the offset log on disk.
>  # Root cause is masked. {{ErrorNotifier.markError}} overwrites previously stored errors, so a later cascading failure (e.g. {{{}concurrentStreamLogUpdate{}}}) can replace the original {{{}Permission denied{}}}/{{{}IOException{}}} and surface as the user-visible {{StreamingQueryException}} cause.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org