You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Jungtaek Lim (Jira)" <ji...@apache.org> on 2026/04/06 05:41:00 UTC

[jira] [Created] (SPARK-56361) Onboard error class for NPE in stream-stream join, with context

Jungtaek Lim created SPARK-56361:
------------------------------------

             Summary: Onboard error class for NPE in stream-stream join, with context
                 Key: SPARK-56361
                 URL: https://issues.apache.org/jira/browse/SPARK-56361
             Project: Spark
          Issue Type: Task
          Components: Structured Streaming
    Affects Versions: 4.2.0
            Reporter: Jungtaek Lim


When there is some issue on stream-stream join (most likely an out of sync between two state stores), stream-stream join can throw NPE.

Throwing NPE does not help anything for users and devs. For users, NPE is mostly cryptic error and they know nothing what to do to mitigate the issue. There is no information around context, so debugging is almost impossible for devs when this happens.

We have captured several places which could throw NPE - we should provide the better error for users which is less cryptic and gives a quick mitigation if they are willing to tolerate losing some data to keep the query running. For devs, we should leave the context to the log so that when users come to devs with the error, devs can at least know where to start with.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org