[jira] [Resolved] (SPARK-56361) Onboard error class for NPE in stream-stream join, with context

["Jungtaek Lim (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-56361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jungtaek Lim resolved SPARK-56361.
----------------------------------
    Fix Version/s: 4.2.0
       Resolution: Fixed

Issue resolved by pull request 55214
[https://github.com/apache/spark/pull/55214]

> Onboard error class for NPE in stream-stream join, with context
> ---------------------------------------------------------------
>
>                 Key: SPARK-56361
>                 URL: https://issues.apache.org/jira/browse/SPARK-56361
>             Project: Spark
>          Issue Type: Task
>          Components: Structured Streaming
>    Affects Versions: 4.2.0
>            Reporter: Jungtaek Lim
>            Assignee: Jungtaek Lim
>            Priority: Minor
>              Labels: pull-request-available
>             Fix For: 4.2.0
>
>
> When there is some issue on stream-stream join (most likely an out of sync between two state stores), stream-stream join can throw NPE.
> Throwing NPE does not help anything for users and devs. For users, NPE is mostly cryptic error and they know nothing what to do to mitigate the issue. There is no information around context, so debugging is almost impossible for devs when this happens.
> We have captured several places which could throw NPE - we should provide the better error for users which is less cryptic and gives a quick mitigation if they are willing to tolerate losing some data to keep the query running. For devs, we should leave the context to the log so that when users come to devs with the error, devs can at least know where to start with.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org